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1.  INTRODUCTION 


The  objective  of  the  study  was  to  develop,  implement,  and 
validate  a compensatory  tracking  task  having  a wide  applicability 
and  sensitivity  to  a number  of  operator  stressors  present  in  the 
operator's  environment.  The  task  was  to  be  designed  for  use  with 
the  type  of  stressors  expected  to  be  utilized  in  a program 
investigating  the  effects  of  visual  countermeasures,  thus  serving 
as  a preliminary  screening  task  for  countermeasure  assessment. 

Task  development  was  to  be  based  on  current  human  operator  modeling 
theory,  to  allow  for  the  isolation  of  the  essential  elements  of 
operator  performance  characterizing  tracking  under  stressed 
conditions . 

This  report  summarizes  the  task  development  effort,  from 
the  initial  design  rationale  to  the  experimental  validation  of  the 
task’s  effectiveness  in  identifying  stressor  effects.  Chapter 
2 provides  a brief  description  of  the  optimal  control  model  of 
the  human  operator,  and  discusses  relevant  applications  of  the 
model  to  understanding  operator  performance  in  a variety  of 
control  and  decision  tasks.  Chapter  3 then  utilizes  the  model 
in  a predictive  capacity  to  design  the  tracking  task,  and 
investigate  task  sensitivity  to  simulated  operator  stressors.  The 
simulation  results  presented  in  this  chapter  provide  the  rationale 
for  the  chosen  task,  and  the  detailed  specifications  for  task 
implementation.  An  experimental  program  for  task  validation 
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is  then  discussed,  with  chapter  4 devoted  to  a description 
of  the  experimental  design  and  protocol,  and  chapter  5 to  a 
presentation  and  discussion  of  the  experimental  results.  This 
chapter  also  describes  the  results  of  a post-experimental  model 
analysis  directed  at  identifying  and  interpreting  stressor 
effects.  Chapter  6 concludes  with  a summary  of  the  major 
findings  of  the  study,  and  recommends  areas  of  future  task 
development  and  application. 

Three  appendices  supplement  the  body  of  the  report. 

Appendix  A provides  a detailed  user-oriented  description  of  the 
Performance  Analyzer  developed  and  used  during  the  course  of  this 
study.  Appendix  B provides  some  background  information  concerning 
the  dynamic  characteristics  of  the  Performance  Analyzer  when 
applied  to  the  chosen  tracking  task.  Appendix  C summarizes 
the  experimental  data  in  tabular  format,  for  convenient  reference. 


2.  THEORETICAL  BACKS.' 'UNO 


The  basic  premise  underlying  the  tracking  task  development 
effort  is  that  a well-trained  and  well-motivated  human  operator 
will  perform  a task  in  a near-optimal  manner,  subject  to  his 
inherent  performance  limitations  and  his  understanding  of  the 
task.  As  we  show  in  this  section,  this  assumption  leads  to 
tractable  mathematical  models  that  not  only  have  predictive 
capabilities  but  can  also  be  used  to  identify  the  underlying 
causes  of  experimentally  observed  differences  in  operator  response 
characteristics . 

In  the  following  discussion  we  describe  relevant  features  of 
the  human  operator  model  and  summarize  some  earlier  validation 
experiments . 

2.1  Human  Operator  Model 

2.1.1  Basic  Model  Structure 

Figure  2.1  illustrates  the  basic  model  structure  for  tasks 
which  require  the  human  to  process  continuous  information.  This 
conceptual  model  is  intended  to  apply  to  a situation  in  which  the 
human  is  required  to  estimate  the  "state"  of  some  process,  on  the 
basis  of  one  of  more  sensory  informational  variables. 

Principal  model  elements  are: 

(a)  A linearized  description  of  the  system  dynamics  given 
by  the  following  state  equation: 

x(t)  = A x(t)  + E w(t)  (2.1) 
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Figure  2.1:  Human  Operator  Model 
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where  x(t)  is  the  vector  which  describes  tne  state 
of  the  system,  and  w(t)  a vector  of  white  driving 
noise  processes.  (If  the  external  forcing  functions 
are  rational  noise  spectra  of  first  order  or  higher, 
the  resulting  "input  states"  are  incorporated  into 
the  state  vector  x(t) ) 

(b)  A "display  vector"  which,  in  general,  consists  of 
a linear  transformation  of  the  state  variables  and 
is  given  as 

Y_(t)  = C x ( t ) (2.2) 

This  vector  contains  the  set  of  sensory  variables 
used  by  the  operator.  Since  both  displacement  and 
rate  information  are  typically  available  from  a 
single  display  element,  the  number  of  elements  in 
the  y vector  can  exceed  the  number  of  physical 
display  elements  used  in  a specific  task. 

(c)  A representation  of  the  pilot's  limitations  by 
means  of  an  equivalent  perceptual  time  delay  t 
and  an  equivalent  observation  noise  vector  Vy(t) 
to  account  for  the  various  sources  of  "processing 
noise"  associated  with  sensory,  central-processing, 
and  response  activity. 

(d)  A predictor  and  estimator  to  provide  the  "best" 
estimate  of  the  current  system  state  in  a least- 
squared-error  sense. 

(e)  An  optimal  response  strategy  acting  on  the  best 
estimate  of  the  state  vector,  the  output  of  which 
is  the  overt  response  activity  u(t) . 
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The  structure  of  the  "optimal  response  strategy"  element  will 
depend  on  the  nature  of  the  task.  In  the  case  of  closed-loop 
continuous  control,  this  clement  will  consist  of  a sot  of  feedback 
gains  to  generate  a control  (or  commanded  control)  input  to  the 
system  (Kleinman  et  al.  (1970);  Baron  et  al.  (1970);  Kleinman  ot 
al.  (1971)) . If  the  task  is  one  of  decision-making,  this  element 
may  consist  of  a Bayesian  decision  process,  the  output  of  which 
is  some  action  appropriate  to  the  decision  reached  (Levi son  anil 
Tanner  (1971)), 

Because  wo  are  concerned  with  closed-loop  tracking  performance, 
the  ensuing  discussion  of  this  model  is  directed  toward  the  task 
of  continuous  control. 

The  model  of  figure  2.1  becomes  a model  for  continuous  control 
as  shown  in  figure  2.2,  by  inclusion  of  the  following:  (1)  an 
optimal  control  strategy,  (2)  a first-order  lag  to  account  for 
apparent  limitations  on  the  operator’s  response  bandwidth,  and 
( >)  a "motor  noise"  process  to  account  for  motor-related  sources 
ot  response  randomness.  To  account  for  the  influence  of  the 
operator’s  control  signal  u(t)  on  system  response,  equation  (2.1) 
is  expanded  as  follows: 

x ( t ) * A x ( t ) + B u ( t ) + E w ( t ) (2.3) 

The  optimal  predictor,  optima)  estimator,  and  optimal  gain 
matrix  represent  the  set  of  "adjustable  parameters"  bv  which  the 
eont roller  tries  to  optimize  his  behavior.  The  general  expressions 
tor  these  model  elements  depend  on  tho  system  dynamics,  according 
to  well-defined  mathematical  rules  that  are  described  in  the 
literature  (see,  for  example,  Kleinman  et  al.  ( 1 7 0 ) ) . The 
controller  is  assumed  to  adopt  a response  strategy  which  minimizes 
the  fol lowing  sum  of  averaged  display  and  control  variances: 
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For  all  of  the  sinqle- variable  control  situations  we  have  explored, 
good  approximations  to  experimental  measurements  have  been  obtained 
usinq  a cost  functional  consistinq  simply  of  a weighted  sum  of 
system  error  variance  plus  control-rate  variance: 

J = (2.4b) 


■ 


The  cost  on  control  rate  represents,  in  part,  a subjective  penalty 
imposed  by  the  controller  on  making  rapid  control  motions.  In 
addition,  this  term  may  account  indirectly  for  physioloqical 
limitations  on  the  pilot's  bandwidth.  The  inclusion  of  such  a 
term  results  in  the  lag  matrix  T,.,  - often  associated  with  the 
neuromuscular  system  - illustrated  in  figure  2.2. 

2.1.2  Information-Processing  Limitations 

The  time  delay,  observation  noise,  and,  to  some  extent,  cost 
functional  parameters  of  the  model  represent  inherent  human 
limitations.  Numerical  values  for  these  parameters  (or  rules 
for  obtaining  such  numerical  values)  must  be  decided  prior  to  the 
analysis  or  adjusted  in  an  iterative  scheme  to  match  operator 
response  characteristics  in  a particular  situation. 

i 

The  time  delay  required  to  match  operator  response  behavior 
usually  lies  in  the  range  of  0.15  to  0.22  seconds;  for  predictive 
applications  of  the  model,  a time  delay  of  0.2  seconds  is  a 
reasonable  choice.  For  single-variable  laboratory  tracking  tasks, 
experimental  data  are  often  well  matched  if  the  relative  cost 
coefficient  on  control-rate  is  adjusted  to  yield  a lag  in  of  0.1 
seconds.  (Because  this  value  is  similar  to  values  that  have  been 
used  in  simplified  models  of  the  human’s  neuromuscular  system,  we 
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refer  to  the  lag  xn  as  the  "motor  time  constant".)  For  more  complex, 
"real  world"  tracking  tasks,  reasonable  values  for  both  control- 
related  and  display-related  cost  coefficients  can  be  obtained  from 
consideration  of  maximum  attainable  (or  allowable)  values  for 
these  system  variables. 


The  "observation  noise"  process  vy(t)  is  intended  as  a 
mathematical  representation  of  the  combined  effects  of  various 
sources  of  randomness  in  the  human's  sensory,  central-processing, 
and  response  mechanisms.  Each  perceptual  variable  used  by  the 
operator  is  considered  to  be  corrupted  by  a Gaussian  "white  noise" 
process  that  is  linearly  uncorrelated  with  the  sensory  input  and 
with  other  similar  noise  processes.  The  perceptual  information 
available  to  the  operator  from  the  display  vector  y is  therefore: 

Yp(t)  =*  £(t)  + Vy(t)  (2.5) 


where  yp(t)  is  the  perceived  variable,  y(t)  the  variable  presented 
by  the  display,  and  vy(t)  a white  noise  process  having  an  auto- 
correlation function 


♦ 

vivi 


(T) 


(2.6) 


or,  equivalently,  a uniform  power  spectral  density  function  of 
Vy/2ir  over  all  frequencies.*  We  shall  refer  to  the  quantity  Vy 
as  the  "autocovariance"  of  the  noise  process. 


Although  not  physically  realizable,  the  notion  of  a white 
noise  disturbance  is  a useful  mathematical  fiction  that  we  adopt 
because  it  provides  a representation  of  human  operator  randomness 
that  is  readily  handled  within  the  framework  of  modern  control  and 

*The  variable  <5  ( t ) is  a "unit  impulse"  function,  located  at  t » 0, 
that  has  zero  width,  infinite  value,  and  an  integral  over  t of 
unity. 
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estimation  theory,  and  because  it  allows  us  to  model  human  control 
behavior  with  considerable  accuracy.  In  effect,  we  are  assuming 
that  the  bandwidth  of  the  observation  noise  process  is  substantially 
greater  than  the  response  bandwidth  of  the  man-machine  system. 


When  the  system  is  designed  so  that  resolution  and  saturation 
effects  in  sensory  and  motor  processes  are  negligible,  the 
observation  noise  covariance  is  found  to  be  proportional  to  the 
mean-squared  value  of  the  displayed  variable  (for  zero-mean 
signals) . Thus,  the  autocovariance  of  the  white  observation 
noise  associated  with  the  ith  display  variable  may  be  represented 
as 


2 

1TPi  E^yi  (t)  * 


(2.7) 


where  is  the  "noise/signal  ratio"  and  has  units  of  normalized 
power  per  rad/sec.  For  situations  in  which  the  signal  y(t)  is  a 
zero-mean  random  process  having  stationary  statistical  properties 
(as  in  the  case  of  most  laboratory  tracking  experiments) , the 


above  expression  may  be  written  as 
2 


(2.8) 


In  general,  when  the  human  operator  is  provided  with  a single 
display  variable,  both  displacement  and  rate  information  will  be 
observed.  Good  agreement  with  experimental  remnant  data  has  been 
obtained  by  assuming  that  the  "noise/signal"  ratios  for  the  position 
and  rate  variables  obtained  from  the  same  display  indicator  are 
identical.  Numerical  values  for  of  .01(i.e.,  -20  dB)  have  been 
found  to  be  typical  of  single  variable  control  situations  although 
smaller  values  have  been  observed  when  operators  attempt  to  control 
unstable  dynamics. 
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The  other  source  of  human  randomness  is  the  motor  noise  Vjn(t)  . 
This  noise  is  also  assumed  to  be  a zero-mean,  Gaussian,  white  noise 
process  with 


V v.  <T)  - V 6(T) 

11  1 


i 


(2.9) 


A motor  noise/signal  ratio,  , of  about  .003  (-25dB)  has  been  found 
to  provide  a good  match  to  experimental  data. 


2.1.3  Model  Outputs 

Once  the  system  has  been  described  and  values  assigned  to 
operator- related  model  parameters,  predictions  can  be  obtained  for 
the  operator's  prediction,  estimation,  and  control  strategies. 
Having  thus  completely  defined  the  operation  of  the  man-machine 
system,  we  can  predict  a variety  of  metrics  related  to  overall 
system  performance  and  to  human  response  behavior. 

Among  the  metrics  that  the  model  can  predict  are:  (1)  mean 
and  variance  of  any  system  variable;  (2)  the  transfer  function 
(gain  and  phase)  relationship  between  any  two  system  variables; 
and  (3)  the  power  spectral  density  of  any  system  variable, 
partitioned  into  a component  linearly  correlated  with  external 
disturbances  and  a "remnant-related"  component  associated  with 
human  operator  response  randomness.  Thus,  the  model  is  capable  of 
generating  response  metrics  that  can  be  compared  with  a variety 
of  measures  extracted  from  experimental  data. 

2.2  Model  Validation  and  Application 

The  human  operation  model  has  been  validated  for  a variety  of 
control  situations  - mostly  single-variable  laboratory  tracking 
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tasks  (Levison  et  al.  (1971);  Kleinman  et  al . (1970);  Kleinman 
et  al.  (1971)),  but  some  complex  tasks  as  well  (Kleinman 
and  Baron  (1971);  Kleinman  and  Killingsworth  (1974);  Kleinman  et 
al.  (1970)).  The  reader  is  especially  directed  to  Kleinman  et  al. 
(1970) , where  it  is  shown  that  detailed  response  behavior  (as 
well  as  mean-squared  error  scores)  can  be  reproduced  by  the  model 
using  values  for  operator  parameters  (time  delay,  motor  lag,  and 
observation  noise/signal  ratio)  that  are  nearly  invariant  across 
control  tasks.  The  tasks  considered  in  that  reference  span  a 
range  of  operator  response  characteristics,  indicating  that  the 
so-called  "operator  parameters"  reflect  primarily  limitations  of 
human-operator  information  processing  that  are  relatively 
independent  of  the  control  task  parameters.  In  particular,  the 
notion  of  treating  response  randomness  (mathematically)  as  an 
observation  noise  process  is  supported  by  the  ability  of  the 
model  to  reproduce  the  spectrum  of  the  stochastic  portion  of  the 
operator's  response  across  a set  of  tasks  in  which  this  spectrum 
varies  substantially. 

The  optimal-control  model  has  been  applied  extensively  both 
as  a tool  for  predicting  continuous  control  behavior  and  as  a 
tool  for  inferring  changes  in  human  response  limitations  resulting 
from  changes  in  the  nature  of  the  control  task  and/or  task 
environment.  Of  greatest  relevance  to  the  tracking  design  effort 
reported  on  here  are  studies  in  which  the  model  has  been  used  to 
account  for  the  influence  of  environmental  factors  on  tracking 
performance . 

Generally  speaking,  studies  of  environmental  factors  have 
led  to  changes  in  model  parameters  that  one  would  expect  from 
the  physical  nature  of  the  "stress".  In  a study  of  simulated 
anti-aircraft  artillery  tracking,  Baron  and  Levison  (1974)  found 
that  the  presence  of  a visual  stress  was  best  accounted  for  by 


increases  in  observation  noise  parameters.  Levison  et  al.  (1971) 
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subjects  had  to  share  attention  among  two  or  more  uncorrelated 
tracking  tasks.  In  the  same  study,  Levison  et  al.  found  that 
the  subjects  would  reduce  their  noise/signal  ratios  if  the  task 
was  sufficiently  difficult  (i.e.,  if  system  error  were  particularly 
sensitive  to  operator  noise) . 

Changes  in  motor- related  parameters  have  also  been  found.  In 
three  separate  studies  of  the  effects  of  vibration  cn  tracking 
performance,  the  presence  of  whole-body  vibration  has  been 
accounted  for  largely  by  an  increase  in  the  motor  noise  parameter 
of  the  model  (Levison  and  Houck  (1975);  Levison  (1976);  Levison 
(1976)).  In  studies  involving  the  pilot's  use  of  motion  cues  in 
roll-axis  tracking,  the  presence  of  motion  cues  was  accounted 
for  by  inclusion  of  motion-related  sensory  variables  in  the 
definition  of  the  "display  vector"  £.  Since  motion  did  not 
represent  a stress,  values  for  noise/signal  ratios  obtained  in 
previous  laboratory  tracking  tasks  were  found  adequate  to  match 
the  data  (Levison  et  al.  (1976);  Levison  and  Junker  (1977)). 

Gai  and  Curry  (1976)  have  used  the  optimal  control  model's 
information  processing  structure  to  analyze  failure  detection  in 
a simple  laboratory  task  and  in  an  experiment  simulating  pilot 
monitoring  of  an  automatic  approach.  In  both  cases,  a step  or 
ramp  was  added  to  the  observed  signal  at  random  times  to  simulate 
an  instrument  failure.  This  produced  a non-zero  mean  value  for 
the  signal  and  for  the  "residual"  (essentially,  the  difference 
between  what  the  subject  saw  and  what  he  expected  to  see) ; failure 
detection  consisted  of  testing  an  hypothesis  concerning  the  mean 
of  the  distribution  of  the  residuals.  The  good  agreement  between 
predicted  and  observed  detection  time  reported  by  Gai  and  Curry 
would  appear  to  confirm  the  applicability  of  the  modeling  approach 
in  these  particular  tasks. 
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The  studies  of  Levison  and  Tanner  and  Gai  and  Curry  provide 
verification  of  the  information  processing  structure  of  the  optimal 
control  model  in  tasks  not  involving  closed-loop  control.  These 
results  taken  in  conjunction  with  the  implicit  validation  provided 
by  the  tracking  data,  provide  strong  support  for  this  model  of 
human  information  processing. 


. 
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3.  TRACKING  TASK  DESIGN 

The  initial  phase  of  this  study  was  devoted  to  the  choice 
of  the  tracking  task  to  be  used  in  measuring  changes  in  human 
operator  performance.  Two  specific  design  objectives  were  called 
for: 

a.  Overall  sensitivity  of  the  task  to  changes  in  operator 
behavior  induced  by  environmental  stressors 

b.  Differential  sensitivity  of  the  task  to  qualitatively 
different  stressors 

The  first  objective  was  chosen  to  ensure  detection  of  a channo  in 
operator  performance  in  response  to  only  slight  changes  in  the 
tracking  task  environment.  The  second  was  chosen  to  provide  the 
capability  of  identifying  changes  in  operator  strategy  associated 
with  specific  changes  in  the  task  environment. 

To  minimize  the  amount  of  trial-and-error  task  design 
associated  with  an  empirically-based  effort,  the  human  operator 
model  of  the  previous  chapter  was  used  in  a predictive  mode,  to 
simulate  tracking  performance  in  a variety  of  conditions.  This 
allowed  for  a rapid  evaluation  of  a number  of  different  task 
designs,  and  for  a detailed  examination  of  task  sensitivity  to 
simulated  stressors.  This  chapter  describes  the  results  of  this 
sensitivity  study,  and  provides  a rationale  for  the  particular 
tracking  task  chosen  for  experimental  validation. 

3.1  Task  Dynamics 

One  of  the  fundamental  determinants  of  operator  tracking 
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performance  is  the  set  of  dynamics  used  in  the  tracking  loop.  If 
the  dynamics  are  particularly  easy  to  control,  then  performance 
tends  to  be  an  insensitive  function  of  task  stress.*  conversely, 
dynamics  which  are  controlled  only  with  great  difficulty  are 
associated  with  performance  changes  which  are  highly  sensitive  to 
task  stressors.  A particular  example  which  demonstrates  this  is 
the  study  of  hypoxia  on  tracking,  conducted  by  Replogle,  et  al 
(1970)  . 

In  their  study,  they  required  the  subjects  to  perform  two 
control  tasks,  singly  and  in  combination.  An  unstable  set  of 
dynamics  was  used  for  one  of  the  tasks,  and  had  the  following 
form: 


P(s) 


X 

s-X 


(3.1) 


The  critical  frequency  X was  adjusted  continuously  according  to 
the  operator's  ability  to  reduce  system  error:  the  smaller  the 
error  score,  the  more  unstable  the  dynamics  became.  No  loop 
input  signal  was  used  in  this  task,  since  the  subject  tended  to 
track  his  own  remnant.  This  type  of  unstable  task,  with  a 
variable  critical  frequency,  has  been  the  subject  of  some  study 
in  the  past,  and  has  been  named  the  "critical  tracking  task"  by 
Jex  et  al  (1966) . 

The  second  control  task  used  in  this  experiment  was  control 
of  third-order  dynamics  typical  of  aircraft  pitch  dynamics.  A 
loop  input  was  applied  directly  to  the  display  to  simulate  a 
pitch  command,  and  thus  this  second  task  simulated  one  of  pitch 
regulation. 
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Replogle  et  al.  found  that  the  levels  of  hypoxia  which  they 
studied  produced  a significant  performance  degradation  for  the 
task  involving  the  unstable  dynamics,  but  had  no  measurable 
effect  on  performance  in  the  pitch-regulation  task.  In  order  to 
explain  these  results,  we  used  the  human  operator  model,  described 
in  the  previous  chapter,  in  a simulation  of  operator  performance 
under  different  stressed  conditions. 


Details  relating  to  the  pitch  dynamics  and  input  signal  were 
not  available  at  the  time  this  analysis  was  performed.  Accordingly, 
we  selected  a set  of  dynamics  from  the  literature  in  order  to  have 
a basis  for  modeling  this  task.  The  dynamics  were: 


P(s) 


13.4  (1  + 2.3s) 
s [ 1 + 0.316s  + 0.085  s2] 


(3.2) 


which  were  those  used  by  Sadoff  and  Dolkas  (1967)  in  their  study 
of  the  effects  of  sustained  acceleration  on  a simulated  pitch- 
regulation  task.  A second-order  forcing  function  was  assumed, 
and  the  input  bandwidth  was  adjusted  to  give  approximately  the 
same  system  error  scores  that  Sadoff  and  Dolkas  obtained  in  the 
absence  of  acceleration  stress. 

In  our  simulation  of  operator  performance  under  different 
stressed  conditions,  we  chose  to  identify  changes  in  stress  with 
changes  in  the  operator's  noise/signal  ratio.  This  model  parameter 
was  chosen  because  it  is  the  parameter  that  we  identify  most 
closely  with  the  operator's  ability  to  process  information,  as 
discussed  in  the  previous  chapter.  We  represented  the  effects  of 
hypoxia,  then,  by  an  increase  in  the  noise/signal  ratio. 
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A white-noise  forcing  function  was  introduced  in  our 
representation  of  the  unstable  dynamics  task  to  simulate  the 
effects  of  system  noise  and  pilot  tremor.  The  pole  location 
V was  held  fixed,  and  the  relation  between  error  variance  ana 
noise  ratio  was  determined.  Figure  3.1  shows  a set  of  curves 
obtained  for  pole  frequencies  ranging  from  1.5  to  3.0  radian/second. 
Also  shown  on  the  same  log-log  plot  is  the  relationship  between 
system  error  and  noise/signal  ratio  for  the  stable  pitch-regulation 
task.  The  error  scores  are  given  in  arbitrary  units;  it  is  the 
percentage  changes  with  noise  ratio  that  are  of  interest. 


One  can  readily  see  from  the  figure  why  stress  has  a much 
ureater  effect  on  performance  with  the  unstable  task.  Consider, 
for  example,  the  task  of  controlling  a vehicle  with  an  unstable 
pole  at  3 rad/sec.  If  the  pilot  allows  his  noise/signal  ratio 
to  increase  from  -26  dB  to  -24  dB,  error  variance  increases 
nearly  three- fold;  an  additional  increase  of  1 dB  in  noise/signal 
appears  to  make  the  task  uncontrollable.  Thus,  the  pilot  must 
work  very  hard  at  this  task  to  keep  the  error  within  reasonable 
bounds.  Anything  that  diminishes  his  information-processing 
capability  will  cause  him  to  relax  his  error  criterion  so  that 
the  task  difficulty  is  reduced  to  a more  manageable  level. 

The  error  score  is  much  less  strongly  dependent  on  the 
pilot's  noise  characteristics  for  the  stable  pitch  task.  Figure 
3.1  shows  that  a 2 dB  increase,  say  from  -22  to  -20  dB,  raises 
the  error  score  by  less  chan  10  percent.  Even  an  increase  as  large 
as  6 dB  raises  the  score  only  by  about  45  percent.  Thus,  if  the 
levels  of  hypoxia  investigated  by  Replogle  et  al.  correspond  to 
an  increase  in  the  pilot's  noise/sinnal  ratio  of  2 or  3 dB,  then 
we  would  expect  to  observe  a siqnificant  performance  degradation 
on  the  unstable  task  and  no  significant  change  on  stable-task 
performance . 

L_ ” — 


This  sensitivity  of  error  score  seen  in  the  unstable  trackinq 
task  thus  arques  for  the  incorporation  of  unstable  controlled 
element  dynamics  in  the  task  desiqn.  In  the  critical  trackinq 
task  of  Jex  et  al  (1966)  mentioned  earlier,  the  subject  is  required 
to  maintain  control  of  an  unstable  plant  while  the  pole  of  the 
instability  is  systematically  increased  until  control  is  lost. 

This  task  has  been  shown  to  produce  qood  reproducibility  from 
day  to  day,  within  subjects,  on  the  basis  of  a relatively  small 
sample  of  trials.  The  critical  trackinq  task  can  be  implemented 
in  many  different  forms  dependinq  on  the  order  of  the  equation 
representinq  the  instability,  the  use  of  a forcing  function,  and 
the  logic  for  assessing  performance. 

Although  a suitably  designed  critical  tracking  task  (or  set 
of  tasks)  miqht  provide  the  required  sensitivity,  it  is  seriously 
deficient  with  regard  to  performance  measurement.  A single 
measurement  is  produced;  typically,  the  value  of  the  plant  pole 
at  which  the  subject  loses  control.  This  pole  value  has  been 
associated  with  the  pilot's  effective  time  delay  (Jex  et  al,  1966). 
However,  any  mechanism  that  serves  to  degrade  information-processing 
(such  as  an  increase  in  noise/signal  ratio)  will  be  interpreted  as 
an  increase  in  pilot  time  delay.  Therefore,  the  critical  task 
does  not  provide  a valid  means  for  generalizing  the  effect  of  a 
task  stressor,  and  extrapolation  from  the  critical  task  to 
performance  in  other  kinds  of  control  tasks  is  tenuous  at  best. 

This  deficiency  can  be  remedied,  however,  by  utilizing  unstable 
dynamics  with  a fixed  instability,  in  conjunction  with  a loop  input 
signal  which  acts  to  continuously  disturb  the  system.  The  fixed 
instability  allows  for  a steady-state  analysis  of  operator  behavior, 
and  the  input  signal  can  be  chosen  for  efficient  off-line  frequency 
analysis  of  operator  performance.  Such  steady-state  frequency 
domain  information  provides  an  abundance  of  performance  metrics. 
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when  compared  with  the  single  critical  frequency  available  from 
the  time-varying  no-input  critical  tracking  task. 

3.2  Loop  Input  Signal 

A loop  input  signal  which  continuously  disturbs  the  plant 
necessitates  continuous  compensatory  control  actions  by  the  human 
operator.  To  avoid  "pre-programmed"  control  actions,  it  is 
important  that  the  input  signal  be  "random-appearing"  to  the 
operator;  by  this  we  mean  that  the  operator  is  unable  to  predict 
the  time  history  of  the  input  signal.  Such  a signal  may  be 
obtained  from  a random-noise  generator,  or  it  may  be  constructed 
from  a sufficient  number  of  deterministic  signals  (e.g.,  sinusoids) 
so  that  the  subject  is  not  able  to  memorize  the  time  pattern  of  the 
input;  otherwise,  he  may  attempt  a "precognitive"  mode  of  tracking 
which  may  differ  considerably  from  the  strategy  adopted  when  the 
input  appears  to  be  random. 

Even  with  a stable  set  of  plant  dynamics,  tracking  becomes 
especially  difficult  when  the  loop  input  is  a wide  bandwidth 
signal.  For  this  reason,  most  experimenters  choose  to  limit  the 
high  frequency  content  by  either  choosing  a step-down  "shelf" 
spectrum  (for  example,  see  Jex  et  al.  (1966)),  or  by  choosing  a 
spectrum  which  continuously  decreases  in  power  at  the  higher 
frequencies  (see  Levison,  et  al.  (1971)).  The  latter  approach 
can  be  realized  by  choosing  the  power  spectral  density  (PSD) 
of  the  input  to  be  a rational  polynomial  function  of  frequency, 
with  the  number  of  poles  exceeding  the  number  of  zeroes.  The 
simplest  such  function  is  a first-order  PSD  function  given  by 

v 2 

•Ms)  = | — | (3.3a) 

s+a 
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The  break  frequency  a specifies  where  the  signal  power  begins  to 
roll-off,  and  the  parameter  K specifies  the  signal's  RMS  level. 
Substituting  jui  for  s,  the  equivalent  frequency  domain  PSD  function 
representation  is 

t(o>)  = — 2 — 2 (3.3b) 

uj  +a^ 

Of  course,  higher  order  PSD  functions  can  be  used  to  provide  more 
rapid  attenuation  of  the  high  frequency  power,  if  deemed  necessary 
for  the  particular  experiment  at  hand. 

3.3  Tracking  Task  Block  Diaqram 

A block  diagram  of  the  tracking  task  is  shown  in  figure  3.2. 

A compensatory  tracking  display  provides  the  subject  with  a measure 
of  system  error  e,  which  he  attempts  to  minimize  by  appropriately 
manipulating  the  control  stick.  The  stick  signal  u is  summed  with 
the  loop  input  disturbance  signal  d,  to  yield  the  plant  dynamics 
command  pc.  The  plant  output  p is  then  fed  back  to  become  the 

display  signal  e.  This  positive  feedback  results  in  an  "outside-in 
display,  necessitating  negative  feedback  compensation  by  the  human 
operator . 

As  noted  in  section  3.1,  the  plant  dynamics  P(s)  are  given 
by 

P(«)  - — (3.4) 

where  the  plant  pole  X remains  fixed  throughout  the  course  of  a 
tracking  run.  The  PSD  function  of  the  loop  input  d is  chosen  to 
be  first-order,  scaled  so  as  to  ensure  that  the  RMS  level  is 
unity • 

*dd<“>  - ~r~2  (3'51 

u)  +a 
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3.4  Sensitivity  of  Trackinn  Task 

For  a complete  specification  of  the  tracking  loop  illustrated 
in  figure  3.2,  we  need  only  specify  the  value  of  the  plant  pole 
X and  the  input  signal  break  frequency  a,  assuming  unity  gains  for 
both  display  and  stick.  The  choice  of  a particular  (X,a)  pair 
should  be  such  as  to  make  the  task  sufficiently  difficult  to  ensure 
adequate  sensitivity  to  imposed  tracking  stressors,  yet  not  result 
in  a task  which  is  uncontrollable  with  only  slight  degradations  in 
the  tracking  environment.  A sensitivity  study  of  tracking 
performance  is  thus  called  for. 

Several  simulations  of  tracking  performance  were  conducted, 
using  the  human  operator  model  of  chapter  2 imbedded  in  the 
tracking  loop  of  figure  3.2.  All  of  the  operator  model  parameters, 
except  for  observation  noise/signal  ratio,  were  fixed  at  "nominal" 
values  consistent  with  levels  associated  with  previous  human 
operator  research:  time  delay  = 0.2  sec,  motor  time  constant 
= 0.1  sec,  and  motor  noise/signal  ratio  negligibly  small.  The 
observation  noise/signal  ratio  was  varied  over  the  course  of  the 
model  runs,  in  an  effort  to  simulate  the  effects  of  a stressed 
environment  (recall  discussion  of  section  3.1). 

The  main  results  of  these  simulations  are  shown  in  figure 

2 

3.3,  in  which  tracking  error  score  oe  is  plotted  against  the 

observation  noise/signal  ratio  (N/S) . Since  changes  in  score 
are  of  greater  interest  than  absolute  levels,  the  scores  have 
been  normalized  for  a more  convenient  graphic  interpretation  of 
the  results;  for  the  first  two  plots,  scores  were  adjusted  to 
yield  unity  error  at  an  N/S  level  of  -23dB,  and  for  the  third 
plot  adjusted  to  yield  unity  error  at  -26dB.  The  score  curves 
themselves  are  parameterized  against  pole  location  and  input 
bandwidth  a. 
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A comparison  of  the  three  plots  shows  how  sensitivity  increases 
with  pole  location  X and  decreases  with  input  bandwidth  a.  The 
moderate  instability  (A=2)  provides  a reasonable  compromise  between 
insensitivity  (A  = l)  and  uncontrollability  (X=4),  and  for  this 
reason  it  was  decided  to  specify  a 2 radian/sec  pole  for  the  plant 
dynamics.  In  order  to  maximize  sensitivity  with  this  chosen  pole 
value  it  was  also  decided  to  specify  an  input  bandwidth  of  0.5 
radian/second.  The  score  curve  associated  with  this  (A, a)  choice 
of  (2,  0.5)  is  labelled  in  the  figure. 

3.5  Differential  Sensitivity  of  Tracking  Task 

To  this  point,  the  trackinq  task  design  effort  has  been 
devoted  to  ensuring  adequate  sensitivity  to  changes  in  stressor 
levels.  It  is  now  of  interest  to  demonstrate  how  the  task  can  be 
used  to  differentiate  between  qualitatively  different  stressors, 
and  to  do  this,  we  aqain  make  use  of  simulation  results  conducted 
with  the  human  operator  model.  The  results  described  here 
investigate  trackinq  performance  associated  with  the  loop  of 
figure  3.2  and  the  (\,a)  task  parameter  values  specified  in  the 
previous  section. 

Performance  sensitivity  to  four  parameters  was  explored: 
time  delay,  motor  time  constant,  observation  noise/sinnal  ratio, 
and  motor  noise/signal  ratio.  To  be  consistent  with  past  studios, 
the  operator  was  assumed  to  use  both  position  and  rate  information, 
and  separate  observation  noise  processes  ("position  noise"  and 
"rate  noise")  were  associated  with  these  quantities.  The  cor  respond i no 
noise/signal  ratios  were  varied  jointly  to  simulate  a change  in  the 
overall  observation  noise/signal  ratio,  and  they  were  varied 
individually  to  represent  selective  decrements  in  the  ahilitv  to 
obtain  and  use  these  informational  quantities.  Baseline  values  for 
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operator- related  parameters  were  as  before:  time  delay  = 0.2  sec, 
motor  time  constant  = 0.1  sec,  observation  noise/signal  ratio  = 

-23  dB,  and  motor  noise/signal  ratio  negligibly  small.* 

Figure  3.4  shows  the  dependence  of  the  tracking  error  variance 

2 2 
o and  the  variance  of  the  rate-of-change  of  control  input  of  to 

changes  in  the  various  model  parameters.  In  each  plot,  the 

left-most  arrow  on  the  abscissa  indicates  the  assumed  nominal 

value  for  that  particular  parameter;  the  next  arrow  indicates  the 

parameter  value  associated  with  a doubling  of  the  nominal  score; 

the  right-most  arrow  indicates  the  parameter  value  associated  with 

a quintupling  of  the  nominal  score.  Thus,  for  example,  a change 

in  observation  noise  from  -23dB  to  -20dB  cannot  be  differentiated 

2 

from  a chance  in  time  delay  from  0.2  sec  to  0.27  sec  when  only  oe  is 
measured;  this  is  because  both  parameter  changes  result  in  an 
identical  doubled  score. 

At  least  one  of  the  parameters,  however,  can  be  differentiated 

from  the  others  by  considering  measures  other  than  tracking  score. 

As  shown  in  figure  3.4,  all  of  the  control  rate  scores  increase 

with  increasing  tracking  scores  except  for  the  case  due  to  changes 

in  the  motor  time  constant  tn.  Here  o?  actually  decreases, 

reflecting  the  lower  bandwidth  of  the  human  operator.  Thus, 

2 

confronted  with  an  increase  in  tracking  score  oe,  the  experimenter 
can  decide  whether  or  not  there  has  been  a change  in  the  motor 


time  constant  by  simply  examining  the  associated  change  in  the 

control  rate  score  o?. 

u 


*The  nominal  value  for  observation  noise/signal  ratio  was  lower 
than  generally  found  in  most  laboratory  tracking  tasks,  but  was 
consistent  with  values  found  in  a study  employing  unstable  system 
dynamics  (Levison  et  al.  (1971)). 
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to  Model  Parameter  Variations 


If  it  is  determined  that  a change  in  the  parameter  is  not 
responsible  for  the  score  change,  then  the  experimenter  is  faced 
with  differentiating  among  the  remaining  parameters.  Figure  3.5 
shows  the  dependence  of  the  normalized  remnant  spectrum  which  is 
obtained  by  reflecting  the  stochastic  portion  of  the  operator's 
control  response  to  an  equivalent  perceptual  disturbance  on  error 
rate,  normalized  with  respect  to  mean-squared  error  rate.  The 
normalized  remnant  spectrum  is  parameterized  against  model 
parameter  values  associated  with  the  nominal  case  and  with  the 
doubled  and  quintupled  error  scores  of  fiqure  3.4. 

A change  in  each  of  the  model  parameters  results  in  a 
distinctive  change  in  the  frequency  dependency  of  the  normalized 
remnant  spectrum.  A joint  increase  in  the  observation  noise/sicmal 
ratios  for  position  and  rate  perception  results  in  an  overall 
increase  in  the  remnant  level  with  no  appreciable  redistribution  of 
power  with  freauency.  On  the  other  hand,  an  increase  in  position 
noise  selectively  increases  low-frequency  remnant,  whereas  an 
increase  in  rate  noise  selectively  increases  remnant  at  high 
frequencies.  An  increase  in  motor  noise  also  increases  high- 
frequency  remnant,  but  the  details  of  the  frequency  dependency 
are  somewhat  different  from  those  induced  by  increased  rate  noise. 
Finally,  an  increase  in  time  delay  results  in  little  remnant  chanqe, 
throughout  the  frequency  range.  Thus,  changes  in  remnant  behavior, 
in  conjunction  with  relative  chances  in  error  and  control-rate 
scores,  allows  one  to  identify  the  parameter (s)  responsible  for  the 
change  in  tracking  performance. 

A partial  cross-check  can  be  provided  by  examining  changes  in 
the  operator's  describing  function.  Figure  3.6  shows  such  changes 
with  model  parameter  changes,  again  using  parameter  values 
associated  with  nominal,  doubled,  and  quintupled  scores.  Changes 
in  position  noise  result  in  little  change  in  the  describing 
function  and  thus  this  measure  serves  to  isolate  the  position  noise 
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Figure  3.6  (cont.):  Describing  Function  Changes  with 

Model  Parameter  Changes 
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parameter  from  the  other  parameters.  In  addition,  an  increase  in 
the  time  delay  results  in  a distinctive  change  in  the  resonance 
peak  (with  respect  to  both  position  and  sharpness)  and  in  the 
behavior  near  resonance;  these  characteristics  could  serve  to 
distinguish  a change  in  time  delay  from  changes  in  the  motor 
time  constant  and  noise  parameters. 


Figure  3.7  provides  a flow  chart  summary  of  this  step-wise 
approach  to  assessing  changes  in  the  different  task  metrics  and 
inferring  probable  changes  in  human  operator  model  parameters. 

The  chart  assumes  that  tracking  scores,  gains,  phases,  and 
normalized  remnant  levels  are  available  to  the  investigator,  and 
illustrates  one  approach  to  differentiating  among  the  several 
parameters.  In  brief,  the  logical  flow  is  as  follows; 

a)  Check  tracking  error.  If  it  has  not  increased,  then, 
by  definition,  no  stressor  is  present,  and  no  parameter 
identification  is  needed. 

b)  If  trackinq  error  has  increased,  chock  control  input 
rate-of-change.  If  it  has  decreased,  the  probable 
cause  is  an  increase  in  the  motoi  time  constant.  A 
drop  in  gain  at  all  frequencies  and  an  increase  in 
phase  lag  at  mid- frequencies  confirm  the  motor  time 
constant  increase.  Any  other  gain/phase  trends 
indicate  multiple  parameter  changes. 

c)  If  control  input  rate-of-change  has  not  decreased, 
check  the  normalized  remnant  spectrum.  Tf  remnant 
levels  have  not  increased,  the  probable  parameter  change 
is  an  increase  in  the  operator's  time  delay.  A drop 

in  gain  at  low  frequencies,  a drop  in  the  resonant 
frequency,  and  a change  in  the  phase  lag  at  high 
frequencies  confirm  the  time  delay  increase.  Any  other 
gain/phase  trends  indicate  multiple  parameter  changes. 
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d)  If  normalized  remnant  levels  have  increased  at  all 
frequencies,  the  probable  cause  is  an  increase  in  the 
overall  observation  noise/signal  ratio.  A drop  in 
gain  at  all  frequencies  and  an  increase  in  phase  lag  at 
mid -frequencies  confirm  the  overall  noise  increase. 

Any  other  gain/phase  trends  indicate  multiple  parameter 
changes . 

e)  If  normalized  remnant  levels  have  increased  only  at 
low  frequencies,  the  probable  cause  is  an  increase  in 
the  position  noise/signal  ratio.  Gain  and  phase 
changes  only  at  low  frequencies  confirm  the  position 
noise  increase.  Any  other  gain/phase  trends  indicate 
multiple  parameter  changes. 

f)  If  normalized  remnant  levels  have  increased  only  at 
high  frequencies,  the  probable  cause  is  an  increase 
in  the  rate  noise/signal  ratio.  A drop  in  gain  at 
all  frequencies  and  an  increase  in  phase  lag  at  mid- 
frequencies confirm  the  rate  noise  increase.  Any 
other  gain/phase  trends  indicate  multiple  parameter 
changes . 

The  results  of  the  sensitivity  analysis  conducted  in  this 
chapter  demonstrate  that  a change  in  a single  operator-related 
model  parameter  results  in  a constellation  of  changes  in  the 
several  performance  metrics  considered.  By  taking  advantage  of 
the  characteristic  differences  in  parameter  change  effects,  we  have 
shown  how  a step-wise  consideration  of  the  different  metrics  can  be 
used  to  isolate  the  particular  parameter  responsible  for  the 
observed  changes.  This  differential  sensitivity  to  parameter 
variations  is  based  on  simulated  operator  performance;  the  exper- 
imental verification  will  be  discussed  later  in  chapter  5. 
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4.  EXPERIMENT  DESCRIPTION 


In  order  to  validate  the  sensitivity  of  the  tracking  task 
described  in  the  previous  chapter,  a set  of  tracking  experiments 
was  conducted  at  BBN  to  compare  tracking  under  different 
environmental  conditions.  There  were  three  primary  objectives 
of  the  experimental  study: 

a)  Determine  the  amount  of  training  needed  to  ensure 
consistent  operator  tracking  performance 

b)  Validate  the  sensitivity  of  the  task  to  a change  in 
the  operator  environment 

c) Validate  the  differential  sensitivity  of  the  task  and 
its  utility  in  isolating  different  stressor  effects 

Since  the  model  simulations  described  in  the  previous  chapter 
assumed  a f ully-trained  subject,  no  apriori  information  was 
available  concerning  the  amount  of  time  necessary  to  train  on 
the  task.  Thus,  the  first  objective  was  directed  at  determining 
an  empirically  based  estimate  of  training  time.  The  second  two 
objectives  were  directed  at  validating  the  simulation  results, 
and  verifying  the  task's  utility  in  detecting  and  isolating 
stressor  effects. 

4.1  Task  Description 

A block  diagram  of  the  tracking  task  used  in  the  experiments 
is  given  in  figure  4.1,  and  is  essentially  the  same  task  which 
was  analyzed  in  the  previous  chapter.  The  subject  is  provided  an 
error  display  on  which  to  base  his  corrective  actions  needed 
both  for  stabilizing  the  unstable  plant  dynamics  and  for  compensating 
against  the  loop  input  disturbance  signal. 


41 


The  tracking  error  was  displayed  on  a 10  cm  wide  by  6 cm 
high  oscilloscope  face,  with  1 cm  horizontal  and  vertical  grid 
lines  providing  a stationary  reference.  The  outside-in  display 
consisted  of  a vertical  bar  2.6  cm  lonq,  which  was  deflected 
laterally  according  to  the  value  taken  on  by  the  display  signal 
e.  The  display  was  centered  in  azimuth  a distance  of 
approximately  60  cm  from  the  subject's  eyes,  so  that  one 
centimeter  of  display  displacement  resulted  in  approximately 
one  degree  of  angular  displacement.  Subjects’  sitting  heights 
were  such  that  the  display  elevation  angle  was  within  10  degrees 
of  eye  level  of  each  subject. 


The  subjects  used  a left-right  force  control  stick  located 
approximately  35  cm  to  the  right  and  25  cm  in  front  of  the  subject; 
an  arm  rest  at  a comfortable  sitting  height  provided  support 
for  the  subject's  arm  and  hand.  Since  only  5 cm  of  the  stick  were 
available  for  grasping,  subjects  used  a combination  of  finger  and 
thumb  grips  for  manipulating  the  stick.  The  stick  stiffness  was 
such  that  one  pound  of  force  applied  to  the  top  end  of  the  stick 
resulted  in  approximately  1 mm  of  lateral  displacement  of  the 
stick.  The  stick  gain  Ks,  shown  in  the  figure  and  used  to 

convert  from  pounds  of  stick  force  to  centimeters  of  plant  command, 
was  set  at  5 cm/pound,  a level  which  provided  adequate  controllability 
without  being  overly  sensitive. 


The  loop  input  disturbance  signal,  d shown  in  the  figure, 
was  constructed  from  13  sinusoids  whose  amplitudes  were  selected 
so  that  the  input  signal  power  spectral  density  (PSD)  approximated 
the  following  continuous  PSD  function: 


* , « (w) 
dd 


2a 
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a)  +a 


(4.1a) 
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The  first-order  form  of  this  PSD  function  was  chosen  on  the  basis 
of  the  simulation  results  of  the  previous  chapter,  as  was  the 
break  frequency: 

a = 0.5  rad/sec  (4.1b) 

The  input  siqnal  amplitudes  were  all  scaled  to  provide  an  RMS 
input  siqnal  level  of  one  cm.  In  order  to  prevent  subjects  from 
learninq  the  input  waveforms  durinq  the  course  of  the  experiment, 
a random  number  generator  was  used  to  vary  the  phase  relationships 
of  the  input  signal  sinusoids  from  one  tracking  run  to  the  next. 
Thus,  the  input  was  "random- appearing"  as  previously  discussed 
in  section  3.1. 

The  plant  dynamics  used  in  the  experiment  were 

P(s)  » e”1 0s  (4.2a) 


The  first  order  instability  was  chosen  on  the  basis  of  the 
simulation  results  of  the  previous  chapter,  as  was  the  pole  value: 

X = 2 rad/sec  (4.2b) 

The  plant  dead-time  tq  was  not  explicitly  specified  to  be  a part 

of  the  nominal  set  of  dynamics,  but  rather  was  due  to  the  digital 
implementation  of  the  tracking  loop.  Its  value  was  determined 
by  an  input-output  system  measurement  described  in  appendix  B, 
and  was  found  to  be 

ig  * 32  msec  (4.2c) 

As  just  noted,  the  tracking  loop  was  implemented  on  a digital 
computer,  with  the  display  and  stick  providing  the  analog 
interfaces  to  the  human  operator.  The  programmable  digital  system, 
developed  specifically  for  human  operator  trackinq  experiments, 
is  named  the  Performance  Analyzer,  and  is  described  in  detail  in 
appendix  A.  In  brief,  it  provides  the  investigator  with  a 
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simplified  means  for  defining  the  desired  tracking  task,  automated 
calibration  of  the  interface  hardware,  real-time  control  of  the 
tracking  experiment,  storage  of  the  tracking  time  histories,  and 
post-experimental  analysis  of  tracking  performance. 

For  this  set  of  experiments,  the  Performance  Analyzer 
software  was  implemented  on  a Digital  Equipment  Corp.  PDP-11/34, 
located  on  site  at  BBN.  The  experimenter  controlled  the  program 
via  a teletype  terminal,  and  monitored  the  display  error  during 
tracking  via  a slave  scope.  Subjects  were  located  in  an  adjoining 
sound-proof  room,  containing  the  scope  display,  control  stick, 
and  an  intercom  for  communicating  with  the  experimenter.  Room 
lightinq  and  display  intensity  were  adjusted  to  comfortable 
levels,  and  remained  fixed  throughout  the  set  of  experiments. 

To  investigate  the  sensitivity  of  the  tracking  task  to 
changes  in  the  task  environment,  two  simulated  stressors  were 
utilized  during  the  course  of  the  experiment.  One  was  directed 
at  simulating  "visual  noise"  imposed  on  the  tracking  display, 
and  consisted  of  a diffused  display  error  bar  presented  on  the 
scope.  By  inserting  a translucent  sheet  between  the  display  grid 
lines  and  the  face  of  the  display,  the  vertical  bar  could  be 
diffused  over  a width  of  approximately  one  cm,  without  affecting 
the  display  reference  provided  by  the  grid  lines.  The  second 
stressed  condition  was  directed  at  simulating  any  stressor  which 
would  act  to  increase  the  operator's  central  processing  time 
(e.q.,  confusion,  ambiguity,  etc.),  and  was  implemented  by  addding 
a time  delay  to  the  plant  dynamics  controlled  by  the  operator. 

The  time  delay  chosen  was  40  milliseconds,  and  was  in  addition  to 
tlie  dynamic  delay  of  32  milliseconds  already  mentioned  above. 
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4.2  Experimental  Protocol 


Six  male  college  students  between  the  ages  of  18  and  29 
years  of  aae  were  used  for  the  experiment.  All  were  right-handed 
and  had  normal  visual  acuity,  although  some  wore  corrective 
lenses . 

A tracking  run  lasted  for  185  seconds,  and  tracking  score 
was  computed  from  the  standard  deviation  of  the  display  error,  oe. 
This  was  computed  over  an  interval  equal  to  the  period  of  the 
imposed  disturbance  signal,  which  was  163.8  seconds.  Since  the 
scoring  interval  was  shorter  than  the  duration  of  the  tracking  run, 
scoring  did  not  begin  until  10.6  seconds  after  run  commencement, 
to  allow  for  subject  "warm-up". 

Three  such  tracking  runs,  interspersed  with  one  minute  rest 
intervals,  comprised  a block  of  runs.  Four  such  blocks, 
interspersed  with  15-minute  rest  intervals  outside  of  the  subject 
room,  comprised  a normal  day’s  tracking  session  for  each  subject. 
The  overall  experiment  was  conducted  over  a month's  time. 

The  experiment  was  conducted  in  three  phases,  consisting  of 
an  initial  training  phase,  an  intermediate  training  phase  on  the 
stressed  conditions,  and  a control  phase  for  comparing  performance 
under  the  different  conditions. 

4 . ?■ . 1 Initial  Training  Phase 

At  the  outset,  each  subject  was  instructed  as  to  the  task 
dynamics  involved,  and  the  objective  of  minimizing  the  tracking 
score.  The  subject  was  then  made  familiar  with  the  task  dynamics 
by  being  given  the  opportunity  to  stabilize  and  control  the 
display  bar  displacement,  with  no  loop  input  disturbing  his 
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control  actions.  This  two-minute  practice  run  was  then  followed 
by  two  tracking  runs  with  the  disturbance  signal  present,  at  a 
half  cm  RMS  level  (half  its  nominal  value) . If  the  subject 


managed  to  control  the  bar  displacement  for  the  entire  length 
of  both  runs,  the  next  run  was  conducted  at  the  nominal  one  cm 
RMS  input  level,  as  were  subsequent  runs.  If  however,  the  subject 
lost  control  of  the  task  at  this  lower  level,  he  was  rerun  at  the 
same  level  until  able  to  successfully  complete  two  such  runs  in 
a row;  at  that  point,  the  input  level  was  raised  to  the  nominal 
one  cm  RMS  value. 

During  the  first  day  of  tracking,  high  and  low  input  signal 
levels  were  chosen  according  to  the  subject's  ability  to  successfully 
complete  a tracking  run.  Thus,  if  a subject  failed  to  complete  a 
run  at  the  one  cm  RMS  input  level,  subsequent  runs  were  conducted 
with  the  half  cm  RMS  input,  until  two  in  a row  were  successfully 
completed,  whereupon  the  subject  was  returned  to  the  one  cm  RMS 
input  level.  After  the  first  day  of  tracking,  this  training 
protocol  was  no  longer  followed;  instead,  if  a subject  failed  to 
complete  a run  at  the  nominal  one  cm  RMS  input  level,  he  was 
simply  rerun  at  the  same  level. 

LwCh  day  of  training  was  begun  with  a 30-second  "warm-up" 
run,  to  ui low  the  subject  to  adjust  mentally  and  physically  to 
the  task.  This  was  followed  by  four  three-run  blocks,  to  yield 
12  runs  per  day.  The  initial  training  phase  consisted  of  four 
such  days,  to  yield  48  completed  tracking  runs  per  subject. 

Throughout  the  first  training  phase,  tracking  score  was 
reported  to  the  subject  at  the  end  of  each  run,  and  the  subject 
was  encouraged  to  modify  his  handgrip,  gaze,  and  tracking 
strategy  in  any  manner  he  felt  which  would  help  minimize  his 
tracking  score.  Each  subject  was  also  allowed  to  view  a plot  of 
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his  run-to-run  score  history,  to  assess  his  progress  in  learning 
the  task.  To  maintain  motivation,  subjects  were  also  made  aware 
of  each  other's  scores. 

Because  of  early  implementation  difficulties  with  the 
Performance  Analyzer,  the  phases  of  the  loop  input  signal  were 
not  varied  from  run-to-run  for  this  initial  training  phase. 

Thus,  the  same  input  signal  was  used  for  all  tracking  runs,  and 
the  subjects  had  the  opportunity  of  "learning"  the  input  over 
the  course  of  training.  However,  this  was  not  felt  to  be  a 
significant  factor  in  determining  task  proficiency,  since 
comparable  scores  were  later  obtained  when  subjects  tracked 
randomized  inputs.  This  issue  will  be  discussed  later  in 
chapter  5. 

4.2.2  Intermediate  Training  Phase 

All  of  the  subjects  participated  in  a second  training  phase 
directed  at  training  them  in  the  three  task  conditions  which 
would  eventually  be  used  for  stressor  comparisons.  To  avoid  the 
input  signal  "learning"  problem  just  mentioned,  all  runs 
conducted  during  this  intermediate  training  phase  utilized  an 
input  signal  which  was  randomized  from  run  to  run.  One  of  the 
conditions  was  designated  the  nominal  condition  and,  except  for 
the  input  signal  randomization,  was  identical  to  that  used  in  the 
initial  training  phase  just  described.  A second  condition  was 
designated  the  diffused  condition  and  consisted  of  the  diffused 
error  bar  display  described  in  section  4.1.  The  third  condition 
was  designated  the  delay  condition,  and  involved  the  plant 
dynamics  time  delay  also  described  in  section  4.1.  Again,  all 
conditions  utilized  a randomized  input  signal  during  this 
intermediate  training  phase. 
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Each  subject  participated  in  three  training  sessions,  with 
each  day's  session  devoted  exclusively  to  one  of  the  three 
conditions.  As  before,  each  session  consisted  of  four  three-run 
blocks,  to  yield  12  training  runs  per  condition  per  subject.  To 
provide  some  balance  for  sequential  learning  effects,  subjects 
were  exposed  to  the  different  conditions  in  a balanced  order, 
according  to  table  4.1.  Note  that  this  is  not  a full-factorial 
design,  since  subjects  were  trained  as  a pair,  on  a given  condition, 


Table  4.1:  Presentation  Order  for  Intermediate  Training  Phase 


Session 

KDH 

SAH 

Subject 

TMV  DMI 

KJW 

LEK 

1 

A 

A 

B 

B 

C 

C 

2 

B 

B 

C 

C 

A 

A 

3 

C 

C 

A 

A 

B 

B 

A:  nominal 

B:  time  delay 

C:  diffuse  display 


As  before,  the  subject  was  instructed  to  minimize  his 
tracking  error  by  any  means  he  felt  appropriate.  If  the 
condition  was  nominal,  the  subject  was  informed  that  the  task 
was  the  same  as  what  he  had  previously  trained  on.  If  the 
condition  was  the  delay  case,  the  subject  was  informed  that  he 
might  notice  some  response  lag,  but  that  he  should  attempt  to 
do  his  best  at  minimizing  his  tracking  score.  If  the  condition 
was  the  diffused  display  case,  the  subject  was  immediately  aware 
of  the  condition  under  which  he  would  be  tracking;  again,  he 
was  simply  told  to  do  his  best  at  minimizing  his  tracking  score. 
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Each  session  began  with  a 30-second  "warm-up"  run  on  the 
condition  which  would  be  used  throughout  that  day's  session. 

After  each  full  185-second  tracking  run,  the  subject  was  informed 
of  his  tracking  score;  as  before,  the  subject  was  periodically 
shown  his  run-to-run  score  history,  so  that  he  could  assess  his 
progress.  If  a subject  failed  to  complete  a tracking  run,  he  was 
given  a short  rest  period  and  then  simply  restarted  on  the  same 
run. 

4.2.3  Control  Phase 

All  of  the  subjects  participated  in  the  third  phase  of  the 
experiment,  directed  at  comparing  tracking  performance  under  the 
three  conditions  on  which  they  had  trained.  The  tracking  task 
and  the  three  conditions  were  identical  to  those  used  in  the 
previous  training  phase,  and  only  the  order  of  presentation  and 
number  of  replications  were  varied.  This  phase  was  completed  in 
a single  session  of  tracking,  consisting  of  four  three-run 
blocks.  The  overall  design  is  given  in  table  4.2,  and  was  chosen 
to  provide  a balance  for  ordering  effects  within  a given  block, 
for  the  first  three  blocks;  the  fourth  block  is  simply  a replication 
of  the  first. 

Experimental  protocol  was  similar  to  that  used  in  the  earlier 
phases,  except  for  the  initial  "warm-up"  period.  For  the  control 
sessions,  the  subject  was  allowed  to  "warm-up"  with  a 30-second 
run  on  each  of  the  three  conditions,  before  starting  the  first 
block  of  runs.  As  before,  if  a subject  failed  to  complete  a run, 
he  was  given  a short  rest  period,  and  simply  restarted  on  the 
same  run. 
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Table  4.2:  Presentation  Order  for  Control  Phase 


A:  nominal 

B:  diffuse  display 

C:  time  delay 
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5.  EXPERIMENTAL  RESULTS 


The  major  results  of  the  various  tracking  experiments  are 
described  in  this  chapter.  Since  most  of  the  results  presented 
here  are  displayed  graphically,  appendix  C provides  the  reader 
with  a more  detailed  tabulated  summary  of  the  tracking  data.  The 
primary  data  obtained  from  the  experiments  ^re  presented  and 
discussed  in  section  5.1,  and  the  results  of  a model  analysis 
effort  are  discussed  in  section  5.2. 

5.1  Primary  Data 

The  primary  data  obtained  from  the  tracking  runs  consisted 
of  tracking  scores  (error,  stick,  and  stick  rate) , describing 
functions  (gain  and  phase) , and  normalized  remnant. 

Tracking  scores  were  calculated  for  each  individual  run 
included  in  the  data  analysis.  These  scores  consisted  of  the 
variance  of  the  display  error,  equivalent  stick,  and  equivalent 
stick  rate.  To  obtain  the  latter  two  scores,  the  stick  signal 
was  first  converted  from  the  recording  units  (lbf)  to  equivalent 
units  of  plant  command  (cm) , via  the  stick  gain  Kg.  The  variance 
of  this  converted  stick  signal  was  then  computed  to  yield  the 
equivalent  stick  score.  Since  the  stick  rate  signal  was  not  saved 
during  a tracking  run,  the  equivalent  stick  rate  score  was  also 
computed  from  the  converted  stick  signal.  This  was  done  by  first 
calculating  the  power  spectrum  of  the  converted  stick  signal, 
multiplying  each  component  by  the  square  of  the  associated 
frequency,  and  integrating  over  frequency,  to  yield  an  estimate 
of  the  stick  rate  variance.  Details  of  this  and  other  score  compu- 
tation procedures  are  given  in  the  Performance  Analyzer  description 
of  Appendix  A. 

Describing  functions  were  calculated  for  each  individual  run 
conducted  during  the  control  phase  of  the  experiment,  after  the 


subjects  had  been  trained  on  the  three  experimental  conditions. 
Describing  function  gain  and  phase  values  for  the  human  operator 
were  calculated  at  each  frequency  contained  in  the  loop  input,  by 
dividing  the  Fourier  transform  of  the  equivalent  stick  signal  by 
the  transform  at  the  display  error.  Since  the  Performance 
Analyzer  introduced  a phase  shift  in  the  measurements  (see 
Appendix  B) , all  human  operator  describing  functions  were 
corrected  for  this  shift. 

Estimates  of  normalized  remnant  were  obtained  by  partitioning 
the  equivalent  stick  spectrum  into  input-correlated  and  remnant- 
related  components,  and,  at  each  loop  input  frequency,  calculating 
the  equivalent  remnant  power  referred  to  display  error  rate,  and 
normalizing  by  the  variance  of  the  display  error  rate.  The  mathe- 
matical details  are  discussed  in  Levison  et  al  (1969),  and  some 
implementation  details  are  given  in  the  Performance  Analyzer 
description  of  appendix  A. 

The  input-correlated  and  remnant-related  components  of  the 
stick  spectrum  were  also  used  to  calculate  the  correlated-to-remnant 
power  ratio  (C/R)  at  each  loop  input  frequency.  If  a C/R  ratio  was 
found  to  be  less  than  6dB,  this  indicated  that  more  than  20%  of  the 
response  power  at  that  frequency  was  uncorrelated  with  the  loop 
input.  Accordingly,  the  corresponding  describing  function  gain  and 
phase  values  were  flagged,  to  assure  that  they  would  not  be  included 
in  the  data  base,  because  of  their  low  reliability. 

Additional  data  processing  procedures  are  to  be  found  in 
the  following  sections  describing  the  results  of  the  three 
experimental  phases. 

5.1.1  Results  of  Initial  Training  Phase 

As  noted  in  the  previous  chapter,  each  subject  successfully 
completed  48  training  runs.  On  the  occasion  when  a subject  lost 
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control  of  the  task,  he  was  simply  rerun,  and  the  data  of  the 
aborted  run  excluded  from  the  data  base  used  for  analysis.  This 
resulted  in  approximately  4%  of  the  data  being  excluded,  in  terms 
of  total  tracking  time  generated  by  the  six-subject  population. 

Because  of  initial  difficulty  with  the  task,  subjects 
occasionally  tracked  at  a loop  input  RMS  level  of  (1.5  cm,  rather 
than  the  nominal  1.0  cm.  To  provide  a fair  base  of  comparison 
with  scores  obtained  under  nominal  conditions,  scores  obtained 
with  a 0,5  cm  RMS  input  were  doubled  under  the  assumption  of 
approximate  operator  linearity.  A total  of  24  such  run  scores 
were  adjusted,  out  of  a total  number  of  288  runs  completed  by 
the  six-subject  population. 

Figure  5.1  shows  tracking  score  dependence  on  run  number, 
averaged  across  the  six  subjects.  Population  means  and  standard 
deviations  are  indicated  by  the  dots  and  bars,  respectively. 

Although  48  runs  per  subject  were  conducted,  the  figure  shows 
a substantial  reduction  in  tracking  score  by  about  the  15th  trial; 
by  the  3(lth  trial,  tracking  score  was  within  20%  of  the  scores 
obtained  at  the  end  of  training. 

The  training  data  of  figure  5.1  also  show  how  the  across- 
subject  variance  in  the  score  decreases  rapidly  with  run  number. 

It  would  appear  that  there  are  two  factors  which  contribute  to 
this  reduction  with  run  number:  differences  between  subjects  in 
terms  of  the  speed  with  which  they  "learn"  the  task,  and,  within 
subjects,  a reduction  of  their  run-to-run  variability  with  training. 
The  first  factor  will  contribute  to  large  across-sub ject  variance 
at  the  beginning  of  training,  since  this  is  where  individual 
learning  curves  show  the  greatest  difference.  The  second  factor 
will  similarly  contribute  to  the  observed  reduction  in  across- 
subject  variance  with  run  number,  since  each  subject  becomes 
progressively  loss  variable  from  run-to-run. 
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5.1.2  Results  of  Intermediate  Training  Phase 

As  noted  in  the  previous  chapter,  each  subject  completed  36 
intermediate  training  runs,  12  on  each  condition.  When  a subject 
lost  control  during  a run,  the  same  procedure  was  used  as  during 
the  initial  training  phase:  the  subject  was  simply  rerun  and 
the  data  of  the  aborted  run  excluded  from  the  data  base.  This 
resulted  in  less  than  2%  of  the  data  being  excluded,  in  terms  of 
total  tracking  time  generated  by  the  six-subject  population. 

It  was  anticipated  that  training  times  under  all  three 
conditions  would  be  considerably  shorter  than  that  seen  during 
initial  training.  This  is  confirmed  by  the  tracking  score 
histories  shown  in  figure  5.2,  illustrating  average  behavior  for 
the  six  subjects  with  run  number,  under  the  three  conditions. 

The  nominal  training  data,  in  fact,  show  no  evidence  of  a 
"learning"  curve,  and  would  appear  to  be  a simple  continuation 
of  the  asymptotic  training  behavior  already  illustrated  in 
figure  5.1,  associated  with  the  initial  training  phase.  Since 
the  data  of  figure  5.1  were  obtained  with  a loop  input  signal  which 
was  never  changed  from  run- to-run,  and  the  data  of  figure  5.2 
were  obtained  with  a signal  which  was  randomized  from  run-to-run, 
it  would  appear  that  "learning"  of  the  input  signal  plays  at  most 
a minor  role  in  determining  performance,  at  least  for  this  task. 

Figure  5.2  also  shows  the  tracking  score  histories  for 
training  under  the  two  stressed  conditions:  diffused  display 
and  time  delay.  In  contrast  with  the  nominal  case,  the  diffused 
and  delayed  conditions  show  a definite  learning  period,  although 
it  appears  to  be  quite  short,  on  the  order  of  5 runs.  Also 
evident  are  the  higher  scores  obtained  under  the  diffused  and 
delayed  conditions,  when  compared  with  the  nominal.  Statistical 
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tests  of  siqnficance  will  be  made  in  the  next  section,  but  the 
apparent  score  increases  shown  in  the  figure  suggest  that  the 
diffused  and  delayed  conditions  can  indeed  be  labelled  as 
"stressed"  conditions. 

5.7.3  Results  of  Control  Phase 

As  noted  in  the  previous  chapter,  each  subject  completed  12 
control  runs,  four  on  each  condition.  When  a subject  lost  control 
during  a run,  the  same  procedure  was  used  as  during  the  previous 
two  experiment  phases:  the  subject  was  simply  rerun  and  the  data 
of  the  aborted  run  excluded  from  the  data  base.  This  resulted 
in  less  than  2%  of  the  data  being  excluded,  in  terms  of  total 
tracking  time  generated  by  the  six-subject  population. 

Tracking  scores  and  frequency  domain  measures  were  calculated 
for  each  tracking  run.  These  measures  were  then  averaged  across 
the  four  replications  completed  by  each  subject  under  each 
condition,*  to  yield  the  means  and  standard  deviations  of  the 
various  measures,  associated  with  each  subject  tracking  under  a 
particular  condition.  With  six  subjects  and  three  conditions, 
this  resulted  in  18  sets  of  statistics.  The  means  were  then 
averaqed  across  subjects,  to  obtain  overall  statistical  measures 
of  performance  with  condition.  To  test  for  significant  differences 
between  conditions,  paired  differences  were  formed  from  corresponding 
subject  means;  these  differences  were  then  subjected  to  a two- 
tailed  t-test  of  significance. 


*Low  reliability  gain  and  phase  measurements  were  excluded  from  each 
subject's  average  if  the  corresponding  C/R  ratio  was  found  to 
be  less  than  6 dB. 
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Figure  5.3  shows  score  variations  due  to  the  differing  task 
conditions:  across-sub ject  means  and  standard  deviations  are 

given  for  display  error  (oe)  , stick  (ou)  , and  stick  rate  (o^,) 
scores,  the  latter  two  given  in  the  equivalent  plant  command  units 
of  cm  and  cm/sec,  respectively.  Although  the  standard  deviations 
are  comparable  across  conditions,  the  score  means  show  a definite 
dependence  on  condition.  Tracking  score  is  raised  by  about  30% 
over  nominal,  when  subjects  track  under  the  diffused  or  delayed 
conditions.  A similar  trend  is  seen  with  the  stick  score,  indicating 
more  operator  effort  under  the  stressed  conditions.  The  stick 
rate  score  shows  a similar  trend,  but  is  not  as  definitive  because 
of  the  larger  variances  associated  with  it. 

To  test  for  significant  differences  between  conditions,  paired 
differences  were  calculated  from  the  corresponding  individual 
subject  means.  The  results  of  a two-tailed  t-test  across 
conditions  are  given  in  table  5.1.  Significance  levels  are  shown 
for  the  three  condition  comparisons;  a level  above  0.05  was 
considered  not  significant  (N.S.). 


Table  5.1:  Significance  Levels  between  Conditions 


condition 
' --^jaair 

score  " 

nominal 

diffused 

nominal 

delayed 

diffused 

delayed 

°e 

.001 

.001 

.05 

<*u 

.001 

.001 

.01 

N.S. 

.001 

N.S. 

The  table  confirms  the  conclusions  to  be  made  from  the 
bar  plots  of  figure  5.3.  The  display  score  is  significantly 
different  across  conditions,  as  is  the  stick  score?  the  stick 
rate  score,  however  indicates  a significant  difference  only  when 
nominal  tracking  is  compared  with  delayed  tracking.  The  table 
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Figure  5.3:  Tracking  Scores  as  a Function  of 

Tracking  Condition  (six  subject  average) 
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also  shows  that  all  the  scores  are  significantly  different  when 
comparing  between  nominal  and  delayed  tasks;  the  nominal-dif fused 
comparison  shows  less  difference,  and  the  dif fused-delayed  even 
less  so. 

Figures  5.4a,  b,  and  c summarize  the  frequency  domain 
measures  associated  with  the  three  tracking  conditions:  nominal, 
diffused  display,  and  time  delay.  Means  and  standard  deviations 
for  the  six-subject  population  are  indicated  by  the  error  bars 
at  each  frequency.  All  phase  measurements  were  corrected  by 
the  addition  of  a 32  msec  lead,  to  compensate  for  the  32  msec 
lag  introduced  by  the  measurements  made  by  the  Performance 
Analyzer  (see  appendix  B) . In  addition,  the  phase  measurements 
associated  with  the  time  delay  case  (figure  5.4c)  were  adjusted 
by  the  addition  of  a 40  msec  lag,  corresponding  to  the  40  msec 
dead- time  present  in  the  plant  dynamics  used  in  this  case.  In 
this  way  the  time  delay  is  associated  with  the  human  operator 
rather  than  the  plant  dynamics,  and  thus  the  time  delay  condition 
simulates  the  effect  of  any  stressor  which  would  serve  to  increase 
the  operator's  processing  time  by  40  msec,  without  affecting  any 
other  aspect  of  operator  behavior. 

The  figures  show  that  operator  strategy  is  similar  under  all 
three  tracking  conditions.  The  gain  data  show  the  integrator- 
like response  at  low  frequencies,  and  the  familiar  resonance  at 
approximately  20  rad/sec.  The  across-sub ject  variance  in  gain 
is  least  in  the  midband,  with  larger  deviations  at  both  ends  of 
the  measurement  spectrum.  The  phase  data  show  a nearly  constant 
lag  in  the  neighborhood  of  40  deg,  at  the  low-  and  mid-frequencies, 
and  then  drop  off  rapidly  in  the  neighborhood  of  the  resonant 
frequency.  Except  for  the  three  highest  measurement  frequencies, 
the  phase  data  are  characterized  by  very  small  across-sub ject 
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variances.  The  normalized  remnant  data  show  the  distinctive 
"washout"  pattern  over  the  measurement  frequencies,  with  a break 
frequency  of  approximately  4 rad/sec.  Fxcopt  at  the  lowest  measure- 
ment f roquencies , where  there  are  fewer  frequencies  over  which  to 
do  remnant  averaging,  the  data  are  characterized  by  very  small 
across-sub ject  variances . 


All  of  these  trends  in  the  means  have  parallels  in  the 
illustrated  model  results  of  chapter  3,  as  a comparison  with 
figures  3.5  and  3.6  will  show.  More  relevant  to  the  object 
of  the  study,  however,  are  the  changes  in  the  means  with  condition. 


Figures  5.5a,  b,  and  c provide  a direct  pairwise  comparison 
of  the  frequency  response  measures  obtained  under  the  three 
different  experimental  conditions.  For  each  figure  the  six- 
subject  means  are  shown,  along  with  the  appropriate  symbol 
indicatina  significance  levels  associated  with  the  differences. 
Means  having  no  associated  significance  symbol  were  found  to  bo  not 
significantly  different,  at  the  (1.05  level. 


Figure  5.5a  provides  a comparison  of  nominal  tracking  with 
tracking  under  diffused  display  conditions.  At  the  low-  and 
mid- f requencies,  operator  gain  is  significantly  lower  with  the 
diffused  display.  In  the  neighborhood  of  111  rad/sec  this  gain 
difference  becomes  insignificant;  at  16  rad/sec,  diffused  display 
conditions  are  characterized  by  a higher  gain,  which  appears  to 
rapidly  drop  off  at  higher  frequencies.  These  high- frequency 
gain  trends  art'  consistent  with  a downward  shift  of  the  resonant 
frequency,  when  going  from  a nominal  display  to  the  diffused 
display . 


FREQUENCY  (rod/*»c) 
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Figure  5.5b:  Frequency  Response  Measures  Obtained  under 
Nominal  and  Time  Delay  Conditions 
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Figure  5.5c:  Frequency  Response  Measures  Obtained  under 
Diffused  Display  and  Time  Delay  Conditions 


The  corresponding  phase  data  show  no  significant  differences 
at  the  low-  and  mid- frequencies . However,  in  the  region  from 
8 to  20  rad/sec,  the  diffused  display  results  in  significantly 
greater  phase  lags.  The  lack  of  a significant  phase  difference 
at  the  highest  frequency  is  consistent  with  a "rejoining"  of 
the  phase  curves  characterizing  both  data  sets. 

Figure  5.5a  also  compares  the  remnant  data  obtained  under 
the  two  conditions.  Although  the  differences  are  slight,  all 
of  the  remnant  means  are  higher  with  the  diffused  display,  and 
almost  half  of  the  differences  are  significant.  The  trends  seen 
are  consistent  with  an  overall  increase  in  the  remnant  level, 
over  the  entire  spectrum. 

Figure  5.5b  provides  a comparison  of  nominal  traek  ^ A’ith 
tracking  under  delayed  conditions.  As  in  the  previous  eu>-e 
the  low-  and  mid-frequency  gains  are  significantly  lowe.  with 
the  time  delay  present.  At  8 rad/sec  the  difference  becomes 
insignificant,  and  above  that  the  delayed  conditions  are 
characterized  by  higher  than  nominal  gains.  Again,  the 
high-frequency  gains  trends  are  consistent  with  a drop  in  the 
resonant  frequency,  when  going  from  nominal  tracking  to  delayed 
tracking. 

The  phase  data  show  significant  differences  throughout  the 
spectrum,  except  at  the  very  lowest  frequency  which  has  associated 
with  it  relatively  large  variance  measurements  (see  figures  5.4a 
and  5.4c),  and  in  the  neighborhood  of  5 to  8 rad/sec.  Although 
not  apparent  from  the  figure,  the  low-frequency  lags  are  less 
with  the  time  delay  present,  and  thus  lead  the  nominal  data 
in  this  region.  The  high-frequency  lags  are  clearly  greater 
with  the  delay  present,  and  thus  the  relative  phase  relations 
are  reversed  from  that  seen  at  the  low  frequencies.  The  5 to 
8 rad/sec  neighborhood  in  which  no  significant  differences  are  seen 
is  the  lead-lag  transition  region. 
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Figure  5.5b  also  compares  the  remnant  data  obtained  under 
the  two  conditions.  At  the  low-  and  mid-frequencies  no  obvious 
trends  are  apparent;  at  the  high  frequencies,  the  delayed  tracking 
condition  is  characterized  by  lower  than  nominal  remnant  levels. 
The  fact  that  only  two  of  the  means  are  (marginally)  significantly 
different  suggests  that  the  time  delay  has  little  effect  on  the 
measured  remnant  levels. 

Figure  5.5c  provides  a comparison  of  the  effects  of  the 
diffused  display  with  those  of  the  time  delay.  The  low-  and 
mid-frequency  gains  show  little  difference,  although  the 
high-frequency  trends  suggest  a more  sharply-defined  resonant 
peak  associated  with  delayed  tracking.  The  phase  data  show  little 
difference  up  to  about  4 rad/sec,  at  which  point  the  increased 
lag  associated  with  the  time  delay  becomes  apparent.  Although 
the  phase  differences  at  the  highest  three  measurement  frequencies 
are  not  statistically  significant  (because  of  the  larger  variances 
characterizing  these  measurements) , the  increasing  lag  with 
delayed  tracking  becomes  substantial. 

Figure  5.5c  also  compares  remnant  data  obtained  under  the 
two  conditions.  Except  for  the  measurement  at  0.5  rad/sec,  all 
of  the  remnant  means  are  higher  with  the  diffused  display.  Since 
more  than  half  of  the  differences  are  significant,  the  remnant 
trends  are  consistent  with  the  notion  of  higher  overall  remnant 
levels  associated  with  the  diffused  display,  when  compared  with 
the  time  delay. 
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5.1.4 


Summary  of  Primary  Data 


This  section  summarizes  the  major  findings  of  the  three 
experimental  phases:  initial  training  phase,  intermediate  training 
phase,  and  control  phase. 

The  initial  training  phase  showed  that  the  most  significant 
reduction  in  tracking  score  occurred  within  the  first  15  trials, 
and  that  by  the  30th  trial,  scores  were  within  20%  of  their 
asympototic  levels.  Across  subject  variance  showed  a similar 
reduction  with  run  number,  due  to  both  across-sub ject  differences 
in  learning  time  needed  for  the  task,  and  due  to  a reduction  of 
individual  run-to-run  variability  with  run  number. 

The  intermediate  training  phase  confirmed  the  asymptotic 
performance  of  the  subjects  on  the  nominal  task,  and  was 
characterized  by  considerably  shorter  training  times  on  the 
two  stressed  tracking  tasks.  Asymptotic  performance  on  the  two 
stressed  conditions  was  reached  in  approximately  five  runs,  with 
tracking  scores  for  both  the  diffused  display  and  time  delay 
conditions  considerably  higher  than  that  achieved  under  nominal 
conditions.  The  use  of  a randomized  loop  input  signal  showed 
little  or  no  difference  when  compared  with  the  use  of  a signal 
which  was  repeated  from  run-to-run,  based  on  the  similar  scores 
achieved  during  this  phase  and  those  achieved  at  the  end  of  the 
initial  training  phase. 
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The  control  phase  provided  additional  measures  of  tracking 
performance,  including  tracking  scores,  describing  functions,  and 
remnant  levels.  Tracking  error  and  stick  displacement  scores 
obtained  under  the  two  stressed  conditions  were  found  to  be 
significantly  higher  than  the  corresponding  nominal  scores,  with 
differences  on  the  order  of  30%.  Stick  rate  score  differences 
were  not  significantly  different,  except  when  comparing  nominal 
with  delayed  tracking. 

The  operator  describing  functions  obtained  during  the 
control  phase  all  showed  the  same  basic  trends  with  frequency, 
for  the  three  different  tracking  conditions.  Within  subject 
t-tests  showed,  however,  that  the  stressed  conditions  resulted 
in  a significant  lowering  of  the  low-  and  mid-frequency  gains; 
the  high-frequency  gain  changes  were  consistent  with  a reduction 
of  the  operator's  resonant  peak  frequency,  combined  with  a 
sharpening  of  the  gain  resonance.  Significant  phase  differences 
were  also  found  when  comparing  the  two  stressed  conditions  with 
nominal  tracking,  the  primary  effect  being  an  increase  in  the 
hiqh- frequency  lag.  Tracking  with  a time  delay  also  resulted 
in  a small  but  significant  lead  at  the  low  frequencies,  when 
compared  with  the  phase  data  obtained  under  nominal  conditions. 

The  normalized  remnant  measures  obtained  during  the  control 
phase  provided  an  additional  dif ferentation  between  the  effects 
of  the  three  tracking  conditions.  The  diffused  display  resulted 
in  consistently  higher  than  nominal  remnant  levels  throughout  the 
measurement  spectrum,  with  almost  half  of  the  differences 
significant.  In  contrast,  the  effects  of  the  time  delay  showed 
no  consistent  pattern  in  the  remnant  measurements,  when  compared 
with  the  nominal  measurements,  and  only  two  of  the  13  measures 
were  found  to  be  significantly  changed  due  to  the  delay. 
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5.1.5 


Discussion  of  Primary  Data 


The  findings  just  summarized  serve  to  verify  the  human  operator 
model  predictions  made  earlier  in  chapter  3,  and,  when  interpreted 
in  terms  of  model  parameter  variations,  they  verify  the  utility 
of  the  task  for  differentiating  between  qualitatively  different 
task  stressors. 

A comparison  of  the  measured  frequency  response  given  in 
figures  5.4a,  b,  and  c with  the  model  predictions  of  figures 
3.5  and  3.6,  shows  that  the  model  predicts  the  basic  describing 
function  and  remnant  trends  with  frequency,  with  a "nominal" 
operator  parameter  set.  Since  the  measured  response  under  the 
stressed  conditions  differs  only  slightly  from  that  made  under 
nominal  conditions  (compare  figures  5.4b  and  c with  figure  5.4a), 
the  implication  is  that  the  model  parameters  need  only  be 
changed  slightly  from  their  "nominal"  values,  to  account  for  the 
effects  of  either  stressor.  That  this  is  indeed  the  case  will 
be  seen  in  the  next  section;  for  now,  however,  it  suffices  to 
note  that  we  should  expect  only  slight  changes  from  nominal, 
with  no  extensive  parameter  variations  needed  to  account  for  the 
stressor  effects. 

Four  basic  features  characterize  the  changes  in  the  frequency 
response  measures  when  going  from  the  nominal  tracking  condition 
to  the  diffused  display  condition  (see  figure  5.5a);  a)  a 
lowering  of  the  low-  and  mid- frequency  gain;  b)  a decrease 
in  the  resonant  peak  frequency;  c)  an  increase  in  the  high-frequency 
phase  lag;  and  d)  an  overall  increase  in  the  remnant  level.  A 
review  of  the  model's  predicted  frequency  response,  given  in 
figures  3.5  and  3.6  of  chapter  3,  shows  that  these  changes  can 
be  interpreted  as  simply  an  increase  in  the  overall  observation 
noise/signal  ratio.  Although  variations  in  other  parameters 
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might  account  for  the  describing  function  changes  (figure  3.6), 
none  of  them  produce  the  observed  overall  increase  in  the  remnant 
level  (figure  3.5).  Thus,  if  one  parameter  were  to  be  identified 
with  the  changes  observed  with  the  diffused  display  condition,  the 
best  candidate  would  be  the  overall  observation  noise/signal 
ratio.*  The  fact  that  an  increase  in  this  noise  level  parameter 
can  qualitatively  account  for  the  changes  seen  with  diffused 
display  tracking  is  intuitively  satisfying,  since  one  might  very 
well  expect  a diffuse  display  to  introduce  visual  "noise"  into 
the  tracking  loop. 

A similar  argument  can  be  made  for  the  observed  changes 
in  the  frequency  response  measures  when  going  from  the  nominal 
tracking  condition  to  the  time  delay  condition  (figure  5.5b). 

Here,  changes  similar  to  the  diffused  display  case  are  seen  in 
the  gain  and  phase  data,  although  one  might  argue  for  an  additional 
sharpening  of  the  resonant  peak  with  the  time  delay  case  (figure 
5.5c).  More  significant,  however,  is  the  fact  that  the  remnant 
level  is  effectively  unchanged  from  its  nominal  level;  thus, 
one  can  eliminate  from  consideration  an  increase  in  the  observation 
noise/signal  ratio.  The  model's  predicted  frequency  response 
(figures  3.5  and  3.6)  instead  argues  for  a change  in  the  operator's 
time  delay,  the  only  parameter  which  accounts  for  the  observed 
describing  function  changes  and  the  lack  of  change  observed  in  the 
remnant  level.  The  fact  that  this  parameter  is  effectively 
identified  from  the  measurements  made  under  the  time  delay  conditions 
clearly  serves  to  verify  this  comparative  approach  based  on  model 
predictions . 

*This  does  not  preclude  the  involvement  of  additional  parameter 
changes,  of  course,  and  this  possibility  is  discussed  further 
in  the  next  section. 
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Althouqh  the  discussion  has  been  somewhat  qualitative,  based 
on  general  "features"  of  the  data,  one  can  conclude  that  the 
tracking  task  is  capable  of  differentiating  between  the  effects 
of  two  different  stressors.  The  approach  requires  only  a brief 
comparison  of  the  frequency  response  measures  with  the  human  operator 
model  predictions,  and  serves  to  isolate  the  model  parameter 
change  most  likely  responsible  for  the  measured  change  under  the 
stressed  tracking  condition.  It  is  anticipated  that  such  an 
approach  would  be  quite  satisfactory  for  a screening  evaluation  of 
tracking  stressors;  however,  it  is  the  intent  here  to  provide  a 
more  rigorous  justification  for  this  approach,  and  the  next  section 
addresses  the  problem  in  a more  quantitative  manner. 

5.2  Model  Analysis 

To  interpret  stressor  effects  on  tracking,  a model  analysis 
effort  was  conducted  using  the  optimal  control  model  of  the 
human  operator.  The  primary  objective  was  to  express  stressor 
effects  in  terms  of  model  parameter  changes,  so  as  to  provide 
a quantitative  basis  for  the  arguments  made  in  the  previous 
section.  A secondary  objective  was  to  demonstrate  the  model's 
utility  in  reducing  the  amount  of  data  needed  to  characterize 
tracking  performance,  by  condensing  a large-dimension  data  vector 
to  a small-dimension  model  parameter  vector. 

5.S.1  Analysis  Procedure 

The  method  for  identifying  model  parameters  was  similar  to 
that  used  in  earlier  study  programs  and  described  by  Levison 
et  al  (1976).  Parameter  values  were  sought  which  would  match  the 
various  metrics  of  the  model  to  the  corresponding  experimentally 
derived  metrics,  which  consisted  of;  1)  performance  scores; 
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2)  describing  function  gain;  3)  describing  function  phase;  and 
4)  remnant  level.  The  matching  was  accomplished  by  minimizing  a 
scalar  cost  function  J which  was  defined  as  the  mean  component 
value  of  a four-dimensional  vector  of  matching  errors: 

j = (Jx  + J2  + J3  + J4)/4  (5.1a) 

whe  re 

T 

J = (Jlf  J2,  J3,  J4)  (5.1b) 


Each  component  of  the  matching  error  vector  corresponded  to 
one  of  the  metric  groups  mentioned  above.  Each  component  was 
computed  from  the  sum  of  the  normalized  squared  differences 
between  model  metric  and  experiment  mean;  normalization  was 
accomplished  by  dividing  the  difference  by  the  associated 
experimental  standard  deviation.  To  illustrate,  the  second 
component  J2  was  computed  from  the  describing  function  gain  errors 
according  to: 
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(5.2) 


where  G^  is  the  mean  experimental  gain  at  the  ith  frequency,  cn 
is  the  associated  standard  deviation,  and  G^  is  the  corresponding 
model  prediction;  for  this  experiment,  N was  set  to  13  to  correspond 
to  the  number  of  test  frequencies  used.  A similar  procedure  was 
used  for  the  other  components  of  the  matching  error  vector  J. 

This  type  of  error  function  provided  for  a simultaneous  matching 
of  the  various  metrics.  The  normalization  procedure  just  described, 
combined  with  the  averaging  of  (5 . 1),  assured  that  an  error  score 
of  unity  was  obtained  whenever  model  predictions  differed,  on 
the  average,  from  experimental  measurements  by  one  standard 
deviation.  Thus,  the  error  function  also  provides  some  qualitative 
indication  of  "goodness-of-f it" . 
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The  model  matching  effort  was  conducted  for  each  of  the  three 
data  sets  obtained  under  the  three  conditions  tested  in  the 
experimental  control  phase:  a)  nominal  task;  b)  diffused  display; 
and  c)  time  delay.  No  attempt  was  made  to  hold  particular  parameter 
values  constant  across  conditions;  instead,  a global  search  of  model 
parameter  values  was  made  for  each  of  the  data  sets,  to  arrive 
at  three  separate  parameter  sets,  one  for  each  condition.  In  this 
way  the  effect  of  a stressor  could  be  directly  interpreted  in 
terms  of  its  effect  on  the  model  parameters. 

The  optimal  control  human  operator  model  used  in  this  analysis 
was  essentially  that  described  in  chapter  2,  with  two  modifications 
to  account  for  low-frequency  phase  trends:  a)  motor  noise  was 
added  to  control  rate,  rather  than  to  commanded  control  as  described 
earlier  in  chapter  2;  and  b) , the  concept  of  "pseudo  motor  noise" 
was  introduced  to  allow  a differentiation  between  the  actual  motor 
noise  driving  the  system  and  the  pilot's  internal  estimate  of  this 
noise.  A detailed  description  of  this  revised  model  is  given  by 
Levison  et  al  (1976) . The  model  was  used  in  a simulation  of  the 
tracking  task  described  in  the  previous  chapter,  with  the  loop 
input  disturbance  signal  power  spectral  density  function  given  by 
(4.1).  The  modelled  plant  dynamics  were  those  described  earlier, 
and  defined  by  (4.2);  the  plant  dead-time  was  simulated  with  a first- 
order  Pade  approximation.  The  model  analysis  provided  performance 
scores,  describing  functions,  and  normalized  remnant  estimates,  in 
addition  to  the  model  matching  errors  described  above. 

5.2.2  Analysis  Results 

Table  5.2  presents  the  parameter  values  obtained  from  the 
model  matching  effort,  for  the  three  experimental  conditions. 

The  last  row  of  the  table  gives  the  resulting  normalized  matching 
score  (computed  according  to  (5.1)),  and  indicates  that  the 
matching  errors  are  not  unreasonable,  being  between  0.5  and  0.7 
standard  deviations. 
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An  increase  in  the  time  delay  is  one  of  the  primary  effects 
seen  with  either  of  the  two  stressed  conditions:  the  diffused 
display  results  in  an  11%  increase  over  the  nominal  condition 
delay,  and  the  time  delay  results  in  a 23%  increase.  Relevant 
to  the  stressor  identification  problem  is  the  fact  that  the 
delay  condition  results  in  an  increase  of  35  milliseconds  in 
operator  delay,  which  corresponds  to  more  then  85%  of  the  simulated 
40  millisecond  delay  used  in  the  actual  tracking  experiment. 

The  model  matching  effort  has  thus  effectively  identified  the 
time  delay  stressor. 

Table  5.2:  Operator  Model  Parameters  for  Three 
Experimental  Conditions 

Parameter  Dimension  j Nominal  Diffused  Delayed 


Time  delay 

Observation  noise 

sec 

0.154 

0.171 

0.189 

(position) 

dB 

-25.0 

-24.0 

-25.0 

Observation  noise 

(rate) 

dB 

-23.0 

-22.0 

-23.0 

Observation  noise 

(overall) 

dB 

-27.1 

-26.1 

-27.1 

Motor  time  constant 

sec 

0.080 

0.080 

0.080 

Motor  noise 

dB 

* 

* 

* 

Pseudo  motor  noise 

dB 

J -30 

-30 

-28 

Matching  score 

- 

0.52 

0.50 

0.67 

* negligible 


The  observation  noise/signal  ratios  show  that  a best  fit  is 
obtained  when  rate  information  is  considered  to  be  a noisier 
variable  than  position  information,  by  a consistent  2 dB  across 
conditions.  If  this  is  translated  into  allocation  of  attention 
between  the  two  information  channels,  then  the  matching  is  closest 
when  position  is  "attended  to"  61%  of  the  time  and  rate  is 
"attended  to"  39%  of  the  time.  The  overall  observation  noise/signal 
ratio  corresponding  to  this  attention  allocation  is  also  shown, 
and  indicates  that  very  low  levels  are  achieved  (-26  to  -27  dB) , 
indicative  of  the  task  difficulty  and  subject  proficiency.  Of 
direct  relevance  to  the  stressor  identification  problem  is  the 
fact  that  the  delay  condition  is  characterized  by  an  observation 
noiso/signal  ratio  identical  to  that  obtained  under  nominal 
conditions.  In  contrast,  the  diffused  condition  results  in  a 
noise  level  1 dB  higher,  entirely  consistent  with  the  intuitive 
notion  that  the  diffused  display  introduces  visual  "noise"  into 
the  trackino  task.  In  this  sense,  the  model  matching  effort  has 
identified  the  diffused  disolay  stressor. 

The  parameters  related  to  the  "motor"  functions  of  the 
operator  model  show  little  or  no  change  with  experimental 
condition,  as  might  be  expected  from  the  nature  of  the  stressors. 

The  motor  time  constant  maintains  a fixed  value  of  (1.08  seconds 
across  conditions,  and  the  motor  noise  is  best  chosen  to  be 
neqligibly  small  (-100  dB)  for  all  conditions.  The  pseudo  motor 
noise  parameter  was  introduced  as  the  result  of  an  earlier  model 
development  effort  (Levison  et  al.  (1976)),  and  primarily  affects 
iho  accuracy  with  which  the  low  freauency  phase  data  ate  matched. 
Although  the  parameter  values  shown  indicate  that  the  delayed 
ondition  is  best  characterized  by  a 2 dB  increase  in  the  pseudo 
not  <tr /signal  ratio,  the  model  matching  score  is  relatively 

to  a change  in  this  parameter.  In  fact,  if  the 

..  . »n.il  r itio  were  lowered  to  the  -30  dB  level  characterizing 
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the  nominal  and  diffused  display  conditions,  the  matching  score 
for  the  time  delay  case  would  only  be  raised  from  (3.6  7 to  (3.68, 
a less  than  2%  increase  in  modelling  error.  Thus,  the  three 
conditions  can  be  reasonably  well  matched  by  an  identical  -30  dB 
pseudo  motor  noise/signal  ratio. 

Although  the  matching  scores  of  table  5.2  give  some  indication 
of  "goodness-of-f it" , a more  graphic  illustration  is  provided  by 
a direct  comparison  of  model  predictions  with  the  experimental 
data.  Figure  5.6  repeats  the  control  phase  performance  scores 
of  figure  5.3;  the  superimposed  open  circles  indicate  the  model 
predictions  obtained  from  the  matching  procedure  just  described. 

All  of  the  trends  with  condition  are  closely  followed,  with 
small  matching  errors:  an  average  error  of  (3.13  standard 
deviation  for  the  nominal  case,  (3.32  for  the  diffused  display, 
and  (3.57  for  the  time  delay. 

Figures  5.7a,  b,  and  c repeat  the  control  phase  frecniency 
domain  data  presented  earlier  in  f inures  5.4a,  b and  c;  the 
smooth  curves  indicate  the  model  predictions  obtained  from  the 
matching  effort.  Again,  all  of  the  trends  are  clearly  followed, 
both  with  respect  to  frequency  and  across  conditions. 

The  figures  show  that  the  model  provides  an  exceptionally 
good  fit  to  the  gain  data,  for  all  three  conditions.  Matching 
errors  were  an  average  (3.4(3  standard  deviation  for  the  nominal 
case,  0.22  for  the  diffused  display,  and  0.63  for  the  time  delay. 
The  model  duplicated  the  low-  and  mid-frequency  gain  drops  seen 
with  the  stressed  tracking  conditions,  and  also  followed  the 
hiqh- frequency  shift  in  the  gain  data.  Although  the  data  are 
sparse  at  the  highest  frequencies,  the  model  indicates  a definite 
trend  toward  a reduction  in  the  resonant  frequency,  and  an  increase 
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5.6:  Comparison  of  Model  and  Experimental 

Performance  Scores  (model  = open  circles) 
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Figure  5.7a:  Comparison  of  Model  and  Experimental 

Frequency  Measures  - Nominal  Conditions 


in  the  sharpness  of  the  resonance,  as  one  progresses  from  the 
nominal  case,  to  the  diffused  display,  and  finally  to  the  time 
delay. 

I 

The  figures  also  show  a close  fit  to  the  phase  data  for  all 
three  conditions.  Matching  errors  were  an  average  0.94  standard 
deviation  for  the  nominal  case,  0.63  for  the  diffused  display  and 
0.63  for  the  time  delay.  The  fact  that  these  matching  errors 
were  larger  than  those  obtained  for  the  gain  data  primarily 
reflects  the  very  small  standard  deviations  characterizing  the 
phase  data,  rather  than  any  inherent  shortcomings  in  the  model 
matching  procedure.  As  seen  from  the  figures,  the  model  duplicates 
the  high-frequency  lag  increases  seen  with  the  stressed  tracking 
conditions,  and  demonstrates  the  expected  rapid  phase  transition 

ri 

in  the  neighborhood  of  the  resonance. 

The  remnant  data  are  perhaps  the  least  well  fit  data  sets. 

Matching  errors  were  an  average  0.62  standard  deviation  for  the 
nominal  case,  0.82  for  the  diffused  display,  and  0.84  for  the 
time  delay.  Although  the  trends  are  all  matched  by  the  model, 
the  low-frequency  remnant  levels  are  overestimated.  This  is 
due  to  the  fact  that  the  low  frequency  data  means  are  characterized 
by  relatively  large  standard  deviations,  and  thus  are  only  lightly 
weighted  in  comparison  to  the  high  frequency  remnant  data.  The 
small  standard  deviations  characterizing  the  high  frequency  data 
also  serve  to  contribute  to  the  relatively  large  matching  errors 
obtained,  in  spite  of  the  large  variances  at  the  ill-fit  low-frequency 
end.  Despite  these  matching  problems,  however,  the  figures 
demonstrate  how  the  overall  remnant  level  is  raised  with  the 
diffused  display  condition,  while  remaining  effectively  unchanged 
with  the  time  delay  condition.  Such  behavior  is  entirely 
consistent  with  the  earlier  observations  made  concerning  the 
observation  noise/signal  ratio  values  obtained  from  the  fitting 
procedure . 
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6.2.3  Discussion  of  Analysis  Results 

The  model  analysis  results  demonstrate  that  a considerable 
amount  of  data  compression  can  be  achieved  by  suitably  adjusting 
the  human  operator  model  parameters  to  fit  the  data.  Since  each 
data  set  consisted  of  three  scores  in  addition  to  gain,  phase 
and  remnant  measurements  at  13  frequencies,  each  data  set 
consisted  of  42  means  and  42  standard  deviations.  By  using  a 
weighted  fitting  procedure  which  takes  into  account  the  standard 
deviations,  the  human  operator  model  effectively  reduced  these 
84  measurements  to  5 specified  model  parameters,  and  managed  to 
fit  the  data  within  approximately  0.5  standard  deviations,  on 
the  average.  Of  course,  some  accuracy  is  lost  with  this 
less-than-perfect  fit,  and  some  information  is  lost  since  the 
model  only  predicts  means  and  not  variances;  however,  the  data 
reduction  is  substantial  and  the  model  provides  a very  efficient 
means  of  data  compression. 

More  important  to  the  stressor  identification  effort,  however, 
is  the  task's  demonstrated  ability  to  provide  the  experimenter  with 
a means  of  differentiating  between  stressors.  The  discussion  of  the 
previous  section  concerned  model  parameter  variations  most  likely 
responsible  for  the  observed  changes  in  operator  behavior;  here, 
the  model  analysis  provided  quantitative  support  for  the 
conclusions  made  earlier  concerning  the  effects  of  the  two  stressors 
studied  in  this  program:  a diffused  display  and  a time  delay. 

Reference  to  table  5.2  shows  that  the  diffused  display  had 
two  effects:  a)  a 1.0  dB  increase  in  the  operator's  overall 
noise/siqnal  ratio;  and  b)  a 17  msec  increase  in  his  time  delay. 

All  the  other  parameters  were  unchanged  from  the  parameter  set 
associated  with  nominal  tracking  conditions.  The  implication  is 
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that  the  diffused  display  not  only  added  "visual"  noise  to  the 
tracking  loop,  but  also  was  responsible  for  increasing  the 
operator's  time  delay.  The  first  effect  is  entirely  consistent 
with  the  characteristics  of  the  diffuser,  and  thus  serves  to 
validate  the  utility  of  both  the  tracking  task  and  the  model 
analysis  in  identifying  stressor  characteristics.  The  second 
effect,  however,  was  not  expected,  since  an  increase  in  a signal's 
noise  level  need  not  imply  a requirement  for  additional  processing 
time.  The  model  analysis  results,  however,  indicate  that  additional 
processing  time  was  indeed  associated  with  the  stressor,  and  that 
it  is  inappropriate  to  assume  a one-for-one  correspondence  between 
a stressor  characteristic  (e.g. , "noisiness")  and  a change  in 
a single  human  operator  model  parameter  (e.g.,  observation 
noise/signal  ratio) . 

Table  5.2  also  shows  that  the  time  delay  had  two  effects  on 
the  operator:  a)  a 35  msec  increase  in  the  operator's  time 
delay;  and  b)  a 2 dB  increase  in  his  pseudo  motor  noise/signal 
ratio.  All  the  other  parameters  were  unchanged  from  the  parameter 
set  associated  with  nominal  tracking  conditions.  As  argued 
earlier,  the  2 dB  noise  increase  is  of  questionable  significance, 
since  it  only  served  to  reduce  the  matching  error  by  less  than 
2%.  Thus,  it  is  not  unreasonable  to  ascribe  this  apparent  increase 
to  the  variance  of  the  data,  rather  than  to  any  significant  change 
in  operator  behavior.  The  35  msec  increase  in  the  operator's  time 
delay,  however,  is  sizeable,  and  corresponds  quite  closely  to  the 
40  msec  delay  added  to  the  loop.  Since  the  data  analysis  procedure 
associated  the  loop  delay  with  the  operator's  response  (recall 
discussion  given  in  section  5.1.3),  the  delayed  trackinq  condition 
effectively  simulated  any  tracking  stressor  which  would  serve  to 
increase  the  operator's  time  delay,  without  affectinq  any  other 
aspect  of  operator  behavior.  The  fact  that  the  model  analysis 
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accounted  for  35  of  the  40  milliseconds,  with  no  call  for  a 
readjustment  of  any  other  operator  parameters,  again  validates 
the  task's  ability  in  identifying  stressor  effects. 

Because  the  model  fitting  procedure  is  highly  non-linear, 
no  variance  estimates  were  available  for  the  derived  model  parameters. 
Thus,  parameter  value  confidence  intervals  could  not  be  generated, 
and  statistical  tests  of  parameter  differences  between  conditions 
could  not  be  performed.  The  conclusions  made  above  are  thus  subject 
to  a more  rigorous  statistical  verification;  however,  one  can  argue 
for  their  validity  based  on  some  of  the  earlier  results  presented 
in  this  chapter.  First,  the  primary  data  comparisons  made  in 
table  5.1  ar.d  figures  5.5a,  b,  and  c show  a large  number  of 
significant  differences  between  conditions,  throughout  the  range 
of  the  many  measurements  available.  Second,  these  differences  do 
not  appear  randomly  through  the  data,  but  rather  follow  trends 
which  are  consistent  with  particular  model  parameter  changes,  as 
argued  in  section  5.1.5,  and  demonstrated  numerically  in  the 
previous  subsection.  Thus,  it  is  not  unreasonable  to  expect  that 
the  significant  differences  present  in  the  primary  data  measure- 
ments would  be  reflected  as  significant  differences  between  the 
inferred  model  parameters,  if  the  model  parameter  variances  were 
comparable.  That  they  are  comparable  is  supported  by  the 
similarity  of  the  primary  data  spread  seen  in  all  three  experimental 
conditions  (figures  5.4a,  b,  and  c) , and  by  the  comparable  fit 
errors  obtained  from  the  model  analysis  (table  5.2).  Thus, 
although  the  parameter  variances  are  not  available  for  firm 
statistical  tests  of  significance,  one  can  make  a fair  argument 
for  the  significance  of  the  observed  parameter  differences 
discussed  above. 


6.  CONCLUSIONS  AND  RECOMMENDATIONS  FOR  FUTURE  DEVELOPMENT 


6.1  Conclusions 

The  principal  results  of  this  study  may  be  summarized  as 
follows : 


1.  Pre-experimental  analysis  using  the  optimal 
control  model  of  the  human  operator  provides 
a rapid  means  of  tracking  task  design,  and 
allows  the  experimenter  to  investigate  the 
effects  of  various  task  structures  and  parameter 
choices  without  the  need  for  an  extensive  experimental 
oilot  study. 

2.  The  use  of  an  unstable  controlled  element  provides 
the  desired  task  sensitivity  to  environmental 
stress,  and  the  incorporation  of  a sum-of-sines 
disturbance  signal  allows  for  the  calculation 

of  important  operator  frequency  domain  measures 
required  for  the  interpretation  of  stressor 
effects.  The  combination  of  an  instability  with 
a disturbance  signal  ensures  task  sensitivity 
and  stressor  identif iability . 

3.  The  minicomputer-based  Performance  Analyzer 
developed  to  implement  the  tracking  task  provides 
the  experimenter  with  a simplified  means  for 
defininq  the  task  parameters,  automated  calibration 
of  the  interface  hardware,  real-time  control  of  the 
tracking  experiment,  storage  of  the  tracking  time 
histories,  and  post-experimental  analysis  of  tracking 
performance . 
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4.  Subjects  learn  the  basic  tracking  task  relatively 
rapidly,  and  in  15  tracking  runs  are  relatively 

close  to  their  final  asymptotic  scores.  After  learning 
the  basic  task,  subjects  require  on  the  order  of  5 
additional  runs  to  reach  approximate  asymptotic  per- 
formance under  the  stressed  conditions. 

5.  The  task  demonstrates  sensitivity  to  both  stressors 
used  in  this  study:  a display  diffuser  which 
degraded  the  target  image,  and  a 40  millisecond 
dead- time  which  adds  a delay  to  the  operator's 
response.  Tracking  scores  under  both  stressed 
conditions  are  significantly  higher  than  that 
obtained  without  the  stressors,  thus  validating 
the  sensitivity  of  the  task. 

6.  The  describing  function  and  remnant  measures 
obtained  from  the  tracking  data  show  significant 
differences  between  experimental  conditions,  and 
verify  the  trends  predicted  by  the  pre-experimental 
model  simulations.  A direct  comparison  between  the 
data  and  the  model  simulations  shows  that  the 
diffuse  display  acts  to  increase  the  operator's 
observation  noise/signal  ratio;  in  contrast, 

the  dead-time  acts  to  increase  the  operator's 
time  delay.  Thus,  the  task  demonstrates  a 
differential  sensitivity  to  the  two  stressors, 
and  the  various  measures  provided  by  the 
Performance  Analyzer  serve  to  isolate  the 
particular  qualities  of  the  stressor. 
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7.  The  results  of  the  quantitative  model  analysis 
validate  the  qualitative  conclusions  regarding 
stress  effects,  based  on  predicted  operator  trends. 

A best  fit  to  the  diffused  display  tracking  data 
was  obtained  by  a 1.0  dB  higher  than  nominal 
observation  noise/signal  ratio,  and  a 17 
millisecond  longer  than  nominal  operator  time 
delay.  These  results  are  consistent  with  a 
"noisy"  visual  image  generated  by  the  diffuse 
display.  The  dead-time  tracking  data  was  best 
fit  with  a 35  millisecond  longer  than  nominal 
operator  time  delay,  again  an  effect  consistent 
with  the  quality  of  the  stressor.  This  quantitative 
analysis  again  confirms  the  task's  capability  for 
differentiating  between  stressors. 

6.2  Recommendations  for  Future  Development 

The  principal  recommendations  for  future  development  are 
as  follows: 

1.  The  capabilities  of  the  Performance  Analyzer  should 
be  extended  to  include  testing  of  multi-axis 
single-display  tracking  behavior.  This  would  allow 
for  the  simulation  of  a more  realistic  display  of 
the  target  and  provide  the  operator  with  the 
additional  visual  cues  which  typify  aircraft 
tracking  situations.  Due  to  its  modular  design, 
the  extension  of  the  Performance  Analyzer  capabilities 
would  involve  a straight-forward  programming  effort; 
it  is  anticipated  that  the  primary  development 
would  be  devoted  to  display  generation. 
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2.  To  further  simplify  operation  of  the  Performance 
Analyzer,  development  should  proceed  on  a "hardwired" 
version  having  a fewer  number  of  user  options, 

and  with  less  user  input  required  for  specification 
of  the  task  parameters.  A microprocessor-based 
instrument  could  be  directly  developed  from  a 
simplified  version  of  the  current  software, 
thus  minimizing  the  amount  of  software  development. 

By  integrating  the  real-time  hardware  (clock, 

A-to-D  converter,  D-to-A  converter)  with  the  analog 
interface  hardware  (control  stick  and  display) , 
a self-contained  microprocessor-based  instrument 
could  be  developed  to  provide  all  of  the  important 
capabilities  currently  available  from  the 
Performance  Analyzer.  By  reducing  the  number  of 
task  specification  options,  a smaller  software 
development  effort  would  be  needed,  and  the 
completed  instrument  would  provide  for  a 
minimum  of  operational  demands  placed  on  the  user. 

3.  Development  should  proceed  on  a method  for  rapid 
automated  identification  of  human  operator  model 
parameters.  Since  parameter  identification  allows 
for  a substantial  reduction  in  the  amount  of  data 
needed  to  characterize  operator  behavior,  a 
capability  for  the  rapid  analysis  of  individual 
tracking  runs  would  be  highly  desirable.  Parameter 
changes  between  single  runs  could  be  used  directly 
in  identifying  stressor  effects  on  individual  runs. 
Individual  run  processing  would  also  allow  for  the 
generation  of  model  parameter  statistics,  a capability 
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which  is  likely  to  be  required  in  situations  where 
the  stressor  effects  are  small  and  within  the  range 
of  individual  subject  differences.  Since  model 
analysis  is  currently  conducted  manually  using  a 
direct  parameter  search  to  minimize  model  matching 
errors,  it  is  anticipated  that  an  automated  identi- 
fication capability  would  be  a direct  extension  of 
the  current  technique.  This  approach  would  minimize 
the  software  development  effort  and  take 
advantage  of  past  experience  in  model  identification. 

4.  An  effort  should  be  directed  toward  correlating 

the  measurements  made  available  by  the  Performance 
Analyzer  with  other  measures  obtained  from 
typical  laboratory  tracking  situations,  in  particular, 
physiological  measurements.  Because  the  Performance 
Analyzer  provides  a rich  array  of  tracking  performance 
measurements  which  complement  the  large  array 
of  physiological  indicies  available  to  the  investigator, 
significant  correlations  between  performance  and 
physiological  state  may  be  uncovered.  The  failure 
of  such  efforts  in  the  past  may  have  been  due  solely 
to  the  paucity  of  the  performance  measurements 
considered.  The  use  of  the  Performance  Analyzer  in 
such  a correlative  study  could  remedy  this  situation, 
by  providing  a wide  spectrum  of  performance 
measurements . 
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Appendix  A:  Performance  Analyzer  Description 
A . 1 . INTRODUCTION 

The  Performance  Analyzer  is  a programable  system  developed  for 
the  study  of  manual  tracking  performance  under  a variety  of 
conditions.  Currently,  the  system  provides  the  investigator  with  a 
simplified  means  for  defining  the  desired  tracking  task,  automated 
calibration  of  the  interface  hardware,  real-time  control  of  the 
tracking  experiment,  storage  of  the  tracking  time  histories,  and 
post-experimental  analysis  of  tracking  performance.  Subsequent 
sections  presented  here  will  provide  a detailed  description  of 
these  various  functions. 

In  its  present  configuration,  the  Performance  Analyzer 
implements  a single-axis  tracking  task,  illustrated  in  figure  A.1. 
The  dashed  region  indicates  the  functions  implemented  by  the 
Analyzer,  and  defines  the  interface  between  the  system  and  the 
human  operator.  The  display  device  provides  a visual 
representation  of  system  error  to  the  human  operator,  and  the 
control  stick  provides  the  operator  with  a means  of  controlling  the 
displayed  error.  If  the  task  is  one  of  disturbance  nulling,  a 
pseudo-random  signal  (d)  is  added  to  the  operator's  stick  signal 
(s),  to  generate  the  plant  command  (pc)  for  the  simulated  plant 
dynamics  defined  as  part  of  the  tracking  task.  If  the  task  is  one 
of  target  tracking,  the  pseudo-random  signal  (t)  is  instead  added 
to  the  plant  signal  (p),  to  generate  the  display  error  (e). 

With  the  exclusion  of  the  display  device  and  the  control 
stick,  the  Performance  Analyzer  is  implemented  entirely  on  a DEC 
PDP-11/Hi  minicomputer,  so  that  most  of  the  functions  shown  in 
figure  A.1  are  implemented  digitally  in  programable  software.  This 
allows  for  considerable  flexibility  in  task  definition  and  loop 
control.  In  addition,  digital  implementation  allows  for  a 
convenient  separation  between  experiment  control  and  subsequent 
performance  analysis,  by  the  use  of  two  separate  software  packages: 


one  named  OCMRUN  for  definition  and  control  of  the  tracking  task, 
and  the  other  named  OCMANL  for  analysis  of  the  stored  tracking  time 
histories.  These  two  software  packages  are  discussed  in  greater 
detail  in  the  following  two  sections. 


A.  2.  TRACKING  TASK  IMPLEMENTATION:  OCMRUN 


i 


This  section  provides  a closer  look  at  how  the  Performance 
Analyzer  implements  the  closed-loop  tracking  task  Just  illustrated. 
A slightly  more  detailed  breakdown  of  the  loop  is  given  in  figure 
A. 2,  which  emphasizes  the  hybrid  analog/digital  nature  of  the 
Analyzer  (digitized  signals  are  indicated  with  an  asterisk).  The 
three  major  components  of  the  Analyzer  are  indicated  by  the  dashed 
lines:  input  interface  consisting  of  the  stick  and  an  analog 
filter;  output  interface  consisting  of  the  display  scope;  and  the 
digital  computer  responsible  for  closing  the  loop.  This  section 
will  discuss  the  digital  portion  of  the  loop,  and  the  reader  is 
referred  to  section  A. 5 for  further  details  concerning  the 
interface  modules. 


The  digital  portion  of  the  loop  diagrammed  in  figure  A. 2 is 
implemented  by  the  use  of  the  program  OCMRUN.  The  program  provides 
three  major  functions: 


a)  definition  of  the  tracking  task  parameters; 

b)  real-time  control  of  the  tracking  task;  and 

c)  calibration  of  the  input  and  output  interface  modules. 

These  three  functions  are  discussed  in  greater  detail  in  the 
following  subsections. 

A . 2 . 1 Definition  of  the  Tracking  Task  Parameters 


OCMRUN  provides  an  interactive  facility  for  the  definition  of 
the  tracking  task  parameters  of  particular  interest  to  the  user. 
Using  appropriate  prompts,  branching  and  parameter  legality  checks, 
OCMRUN  can  guide  the  user  through  a rapid  "set-  up"  of  the  desired 
task  parameters.  A detailed  example  of  a particular  "set-up"  is 
given  in  section  A. *1.1.  Here,  the  discussion  will  concentrate  on 
the  general  parameter  selection  options  available  to  the  user,  and 
the  checks  made  on  particular  values  selected  by  the  user. 
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OUTPUT  INTERFACE  /-INPUT  INTERFACE 


Single-Axis  Tracking  Loop:  System  Breakdown 


A. 2.  1.1  Time  Base  Parameters 


OCMRUN  prompts  the  user  to  enter  the  sample  time,  Ts  defining 
the  basic  sample  period  of  the  digital  system,  and  the  run  length 
Tr,  defining  the  duration  of  the  tracking  run  to  be  conducted. 
OCMRUN  provides  checks  to  ensure  that  the  user  has  not  entered  too 
small  a value  for  T<.  nor  too  large  a value  for  TR . The  former 
ensures  that  the  sample  period  is  greater  than  the  clock  resolution 
used  for  basic  loop  timing  and  the  latter  ensures  that  the  number 
of  samples  Ns,  where 

NS  = VTs  + 1 (A  .2.1) 


does  not  exceed  the  system’s  preset  upper  limit. 

Given  the  total  number  of  sample  points  Ns  comprising  a run, 
OCMRUN  specifies  the  total  number  of  sample  points  Nq  for  one 
period  of  the  loop  input  signal.  For  compatibility  with  the  FFT 
routine  used  by  OCMANL  (see  section  A. 3),  the  value  for  Nq  is 
chosen  to  be  the  largest  power  of  2 less  than  or  equal  to  Ng.  This 
value  for  Nq  then  determines  the  loop  input  signal  period  Tq,  since 


This  value  also  determines  the  time  period  over  which  run  scoring 
is  conducted  (see  section  A. 2. 1.5). 


A. 2. 1.2  Input  Signal  Parameters 

OCMRUN  questions  the  user  as  to  whether  or  not  a pseudo-random 
input  signal  is  to  be  injected  into  the  tracking  loop.  If  no 
signal  is  desired,  the  program  skips  to  the  next  section  defining 
the  plant  dynamics  parameters.  If  the  user  requests  a loop  input, 
OCMRUN  prompts  the  user  to  specify  either  a target  input  (t*)  or 
disturbance  input  (d*);  this  choice  is  illustrated  by  the  switch 
setting  of  figure  A. 2. 
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The  pseudo-random  input  signal  n*  is  a digital  version  of  the 
following  continuous  sum-of-sines  signal  n: 


n(t)  = T A.  sinfu.t  + A.) 
j-1  J =>  3 


(A. 2.3) 


Several  parameters  must  be  specified  to  define  this  function. 
After  a prompt  from  OCMRUN,  the  user  can  request  the  "nominal"  set 
of  frequencies,  which  means  that  OCMRUN  will  automatically  define 
the  number  of  components  N and  each  frequency  ^ (j=1,N), 
according  to  a preset  table.  Alternatively,  the  user  can  type  in 
his  own  set  of  frequencies;  in  this  case,  OCMRUN  allows  for 
corrections  to  be  made  during  data  entry,  and  provides  checks  to 
ensure  that  the  chosen  frequencies  are  consistent  with  the 
previously  chosen  sample  and  run  times. 

Once  the  frequency  set  has  been  specified,  OCMRUN  provides 
adjustments,  as  necessary,  to  ensure  that  all  of  the  frequencies 
are  integral  multiples  of  a base  frequency  u>0>  or 

= hj  w0  •••/  N)  (A. 2. 4) 


(A.  2. 4) 


where  the  base  frequency  is  related  to  the  period  Tq  of  the  loop 
input  signal  by: 


“0  = 2ir/To 


(A. 2. 5) 


At  the  user's  option,  the  harmonic  multipliers  hj  above  can  be 
specified  to  be  either  simple  integers  or  primes. 

Under  normal  circumstances,  the  user  will  request  that  the 
loop  input  n consist  of  more  than  a single  sine  wave  (N>1).  In 
this  situation,  the  amplitudes  Aj  are  chosen  so  that  the  power 
spectral  densitv  (PSD)  of  n approximates  either  a first  or 
second-order  continuous  PSD  function  of  the  following  form: 

A 

*nn  (w)  = /“5 9)  (A. 2. 6) 


a +w 
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where  X is  chosen  by  the  user  to  be  either  1 or  2,  after  a prompt 
from  OCMRUN.  In  addition,  the  user  is  requested  to  enter  the  PSD 
break  frequency  a and  the  desired  root-mean-square  (RMS)  value  of 
the  loop  input  n.  Given  these  parameters,  OCMRUN  then  proceeds  to 
calculate  the  individual  which  will  satisfy  all  of  these 
cond itions . 

Should  the  user  request  that  the  loop  input  consist  of  only  a 
single  sinusoid  (N=1),  then  OCMRUN  prompts  the  user  for  the  desired 
RMS  value  for  the  loop  input,  and  calculates  the  single  amplitude 
A,  accordingly.  In  this  case,  no  attempt  is  made  to  approximate  a 
desired  PSD  function. 

The  relation  between  the  specified  RMS  value  of  the  loop  input 
and  the  resultant  RMS  display  error  depends  on  the  type  of  tracking 
task  requested  by  the  user.  If  the  task  is  target  tracking  and  no 
stick  signal  is  present,  then  the  RMS  display  error  (in  cm)  will 
equal  the  user-specified  RMS  value  of  the  loop  input.  If  the  task 
is  disturbance  nulling,  the  resultant  RMS  display  error  will  depend 
on  the  plant  dynamics  specified  by  the  user  (see  next  section  for 
options) . 

The  final  set  of  parameters  needed  to  define  the  loop  input  n 
are  the  phases  <p j . By  using  a random  number  generator,  values 
are  chosen  so  as  to  be  uniformly  distributed  between  0 and  2tt. 
The  "seed"  for  the  random  number  generator  is  chosen  on  the  basis 
of  the  particular  run  number  associated  with  the  upcoming  run,  thus 
providing  a consistent  means  of  randomizing  the  phases  from  run  to 
run.  This  ensures  that  the  subject  will  not  "learn"  the  input  over 
a number  of  repeated  runs,  since  the  time  history  of  the  loop  input 
will  differ  for  each  run. 


A. 2.  1.3  Plant  Dynamics  Parameters 

OCMRUN  prompts  the  user  to  specify  the  type  of  plant  dynamics 
P(s)  to  be  controlled  by  the  subject.  Table  1 lists  the  six 
options  currently  available,  ranging  from  a simple  gain  to  a 
second-order  low-pass  filter. 


Type 

1 

2 

3 

4 

5 

6 


Table  1:  Plant  Dynamics  PCs) 


PCs) 

K 

K/s 
K/s2 
K/ (s+a) 

K/s  (s+a) 

Ku>2/(s2+2t  co  s+u)2) 
n n n n 


description 

gain 

integrator 
double  integrator 
lag 

lag  plus  integrator 
2nd  order  low-pass 


Once  the  type  is  chosen,  OCMRUN  prompts  the  user  to  enter  the 
appropriate  plant  parameters.  For  a gain,  integrator,  or  double 
integrator,  only  the  gain  is  required;  for  a lag  or  lag  plus 
integrator,  the  gain  and  break  frequency  are  required;  for  a 
second-order  low-pass  filter,  the  gain,  natural  frequency,  and 
damping  ratio  are  required.  These  parameter  values  are  later  used 
bv  OCMRUN  to  calculate  coefficients  for  a general-purpose 
difference  equation  used  for  the  real-time  digital  simulation  of 
the  chosen  plant  dynamics. 


In  addition  to  the  dynamics  specified  in  table  1,  the  user  has 
the  option  of  specifying  a dead-time  to  be  added  to  the  plant 
dynamics.  After  specifying  the  dynamic  parameters,  the  user  is 
prompted  to  enter  the  desired  amount  of  dead-time,  TD.  A zero 
response  results  in  no  dead-time  added  to  the  plant  dynamics.  A 
non-zero  response  causes  OCMRUN  to  recalculate  the  dead-time  as  the 
nearest  integer  multiple  of  half  the  previously-specified  sample 
time  Tgj  this  calculated  value  is  then  displayed  to  the  user  for 
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confirmation.  A check  is  also  made  to  ensure  that  the  number  of 
dead-time  wait  counts,  Tq/T<.,  does  not  exceed  the  system’s  preset 
upper  limit. 

A.2.1.H  Stick  Gain  Parameter 

OCMRUN  prompts  the  user  to  enter  the  stick  gain  Kg,  used  to 

convert  the  stick  value,  expressed  in  pounds,  to  an  equivalent 

2 

olant  command,  expressed  in  cm,  cm/sec,  or  cm/sec",  depending  on 
plant  order.  Adjusting  this  gain  allows  the  user  to  adjust  stick 
sensitivity  without  changing  the  plant  gain. 

A. 2. 1.5  Scoring  Parameters 

For  each  loop  variable  of  interest,  tracking  scores  are 
calculated  to  yield  estimates  of  the  mean,  RMS  value,  and  standard 
deviation.  Scoring  is  conducted  over  an  interval  of  length  Tq, 
equal  to  the  period  of  the  loop  input  signal  (recall  section 
A .2.1.2),  and  commences  at  a time  Tj  during  the  run.  Normally,  Tj 
is  set  by  OCMRUN,  but  the  user  is  provided  the  opportunity  of 
specifying  the  scoring  start  time;  if  the  user  chooses  to  do  so, 
OCMRUN  ensures  that  is  small  enough  so  that  the  scoring  interval 
Tq  will  be  over  prior  to  run  termination. 

At  the  end  of  a run,  OCMRUN  also  computes  the  overall  tracking 
score,  based  on  a weighted  sum  of  display  error  variance  and  stick 
signal  variance: 

j = (o2  + W o2) 1/2  (A. 2. 7) 

e u 

The  weighting  parameter  W is  user  specified. 

A. 2. 1.6  Parameter  Files 

Once  the  above  parameters  have  been  specified  by  the  user, 
OCMRUN  provides  the  user  with  the  opportunity  of  storing  them  in  a 
parameter  file,  for  possible  later  use.  In  this  way  the  user  can 
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create  several  different  parameter  files  for  different  tracking 
tasks  of  interest.  By  doing  this  prior  to  the  actual  tracking  run, 
the  user  can  then  minimize  "set  -up"  time  by  simply  reading  in  the 
appropriate  parameter  file  at  run  time,  rather  than  proceeding 
through  a specification  of  each  individual  task  parameter.  OCMRUN 
supports  both  reading  from  and  writing  into  user  specified 
parameter  files,  and  provides  the  user  with  the  appropriate  prompts 
for  guidance  during  file  I/O. 

A . 2 . 2 Real-Time  Control  of  the  Tracking  Task 

OCMRUN  provides  real-time  control  of  the  tracking  task  by 
proper  sequencing  of  the  computations  needed  to  implement  the  loop 
illustrated  in  figure  A. 2,  and  by  the  timing  facility  afforded  by 
the  system's  digital  clock.  In  addition,  some  housekeeping 
functions  are  maintained  bv  OCMRUN,  to  aid  the  user  in  run 
identification  and  data  file  labelling.  Detailed  examples  of  these 
housekeeping  functions  and  the  subsequent  real-time  output  are 
given  in  section  A.4.1;  here, the  discussion  will  concentrate  on  the 
general  features  available  to  the  user  for  conducting  a tracking 
task . 

A. 2. 2.1  Run  Identification  and  File  Labelling 

Prior  to  the  start  of  an  actual  run,  OCMRUN  displays  to  the 
user  the  current  date  and  the  run  number  associated  with  the 
upcoming  run.  The  user  is  then  provided  the  opportunity  of 
changing  the  run  number,  if  he  so  desires.  Both  the  data  and  run 
number,  in  addition  to  the  current  clock  time,  are  subsequently 
stored  with  the  data,  to  aid  in  run  identification.  OCMRUN  then 
prompts  the  user  for  comments  to  be  stored  with  the  data  of  the 
upcoming  run;  these  partially  fulfill  the  function  of  a laboratory 
notebook  in  providing  additional  information  concerning  the 
particulars  of  the  run. 
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OCMRUN  then  prompts  the  user  for  a disc  file  name  to  be  used 
for  data  storage.  Some  simple  checks  are  made  concerning  the 
legality  of  the  name,  and  if  found  valid,  the  name  is  used  to  open 
a data  file  on  the  disc.  This  file  is  then  immediately  used  to 
store  the  run  identification  and  file  labelling  information  just 
entered  by  the  user.  In  addition,  all  of  the  parameter  values 
defining  the  particular  tracking  task  are  also  saved  in  the  file. 
This  provides  the  user  with  the  later  option  of  listing  all  those 
parameters  discussed  in  section  A. 2.1,  for  a firm  identification  of 
the  task  parameters  used  during  a particular  run.  It  also  allows 
for  a simple  restart  procedure,  as  described  later  in  section 
A .2.2.4. 

A. 2. 2. 2 Real-Time  Task  Control 

Once  the  above  file  maintenance  functions  have  been  completed, 
OCMRUN  proceeds  to  initialize  various  routines  to  be  used  in  the 
conduct  of  the  tracking  task.  This  results  in  a zeroing  of  the 
plant  initial  conditions,  initialization  of  the  loop  input  signal 
generator  and  data  filing  routine,  and  zeroing  of  the  cumulative 
scoring  counters.  In  addition,  the  output  D-to-A  line  is  zeroed  to 
provide  the  subject  with  an  initially  zeroed  display. 

With  this  initialization  completed,  OCMRUN  prompts  the  user 
for  the  proper  terminal  entry  to  begin  the  run.  This  entry  begins 
the  basic  computational  loop  responsible  for  real-time  control  of 
the  tracking  experiment. 

The  loop  begins  with  an  A-to-D  conversion  of  the  subject's 
stick  signal  (recall  figure  A. 2).  Using  the  scale  factors  obtained 
from  an  earlier  calibration  (see  section  A. 2. 3),  the  stick  voltage, 
s,  is  converted  to  units  of  pounds  force  and  then,  using  the  stick 
gain  factor  Kg,  to  units  of  equivalent  plant  command  (cm,  cm/sec, 
or  cm/sec^,  depending  on  plant  order).  If  the  task  is  one  of 
disturbance  nulling,  this  equivalent  stick  signal  u*  is  then  summed 


with  the  signal  d*  from  the  loop  Input  signal  generator,  to  yield 
the  plant  command  pc*  • If  the  task  1 si  one  of  target  tracking,  or 
If  user  has  specified  that  no  loop  Input  be  used,  then  the  plant 
command  p0*  is  simply  set  equal  to  the  equivalent  stick  signal  u*  . 

The  digital  filter  P(  which  simulates  the  desired  plant 
dynamics  is  then  updated,  based  on  the  current  and  past  plant 
commands,  and  the  current  and  past  plant  state.  The  updated  state 
is  then  used  to  calculate  an  updated  version  of  the  plant  output 
p* . If  the  task  is  one  of  target  tracking,  the  plant  output  is 
then  summed  with  the  signal  t*  from  the  loop  input  signal 
generator,  to  yield  the  display  error  signal  e* . If  the  task  is 
one  of  disturbance  nulling,  or  if  the  user  has  specified  that  no 
loop  input  be  used,  then  the  display  error  signal  e*  is  simply  set 
equal  to  the  plant  output  signal  p* . Using  a scale  factor  obtained 
from  an  earlier  calibration  (see  section  A. 2.1),  the  display  error 
signal  e#  is  then  converted  to  a display  voltage  signal,  e,  via  a 
U-to-A  conversion. 

Once  the  display  error  signal  has  been  generated  by  OCMRUN,  no 
further  real-time  operations  are  necessary  until  the  next  stick 
sample  is  to  be  made.  This  "wait."  time  is  used  by  OCMRUN  for 
several  purposes.  The  loop  variables  of  interest  ( n*  , s*  , p* , and 
e* ) are  appropriately  scaled  and  then  stored  in  an  output  data 
buffer  for  later  filing  on  the  disc  data  file.  Cumulative  scores 
for  these  variables  are  also  calculated,  to  maintain  running  sums 
and  running  sums  of  squares.  The  appropriate  value  for  the  loop 
input  signal  at  the  next  sample  time  is  calculated,  in  anticipation 
of  its  use  over  the  next  computation  Interval.  Finally,  the  plant 
dynamics  are  updated  to  reflect,  the  newly  generated  state 
var i ables . 

Once  these  functions  have  been  completed,  OCMRUN  enters  a wait 
loop,  periodically  checking  the  elapsed  time  indicated  by  the 
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system's  real  time  clock.  The  elapsed  time  is  measured  from  the 
time  of  the  last  A-to-D  conversion  which  began  the  loop 
computations;  once  it  reaches  a pre-set  value  determined  by  the 
user-specified  sample  period,  OCMRUN  exits  from  its  wait  loop  and 
initiates  another  A-to-D  conversion,  thus  restarting  the  real-time 
loop  computations  just  described.  This  looping  continues  until  the 
desired  number  of  samples  have  been  obtained,  at  which  point  OCMRUN 
proceeds  to  ''close-out”  the  run. 

A. 2. 2. 3 Run  Termination 

At  the  end  of  a run,  OCMRUN  stops  the  real-time  clock,  outputs 
to  the  disc  any  tracking  data  left  over  in  the  output  data  buffer, 
closes  the  disc  data  file,  and  zeroes  the  D-to-A  channel,  thus 
zeroing  the  display.  In  addition,  the  cumulative  scores  obtained 
during  the  run  are  used  to  calculate  the  tracking  scores  for  the 
pertinent  loop  variables  (n*,  s* , p*  and  e*l.  These  scores  consist 
of  each  signal's  mean,  standard  deviation,  and  RMS  value,  and  are 
displayed  to  the  user  via  the  terminal.  The  total  tracking  error 
score  is  also  displayed,  based  on  the  display  score,  stick  score, 
and  stick  score  weighting  factor. 

A summary  printout  of  any  stick  or  display  overloads  which  may 
have  occurred  during  the  run  is  also  provided,  to  notify  the  user 
of  over-voltage  conditions  at  the  A-to  D and/or  D-to-A  converter. 
The  summary  specifies,  for  stick  and  display,  the  percentage 
overload  during  the  scoring  interval,  and  the  point  at  which  the 
first  overload  occurred. 

A. 2. 2. 4 Multiple  Tracking  Runs 

With  a run  completed  and  the  tracking  data  filed,  OCMRUN 
questions  the  user  as  to  whether  or  not  another  run  is  desired.  A 
negative  answer  results  in  program  termination  and  a return  to  the 
operating  system  monitor.  An  affirmative  answer  resu  Is  in  a 
continuation  of  the  program  and  a prompt  to  the  user  for  any 
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changes  in  the  tracking  task  parameter  set.  If  no  changes  are 
requested,  OCMRUN  begins  another  sequence  of  run  identification, 
file  labelling,  real-time  control,  and  run  termination,  as  just 
described  in  the  previous  three  sections.  If  the  user  requests 
that  changes  be  made  prior  to  the  upcoming  run,  he  is  given  the 
option  of  reading  in  an  entirely  new  data/parameter  file  (recall 
section  A. 2. 1.6),  or  of  simply  changing  individual  values  of  the 
parameter  set  used  in  the  last  run. 

Once  the  new  parameter  set  has  been  defined,  OCMRUN  increments 
the  run  number  by  one,  and  uses  it  to  generate  a new  set  of 
pseudo-random  phases  for  the  loop  input  signal,  if  one  is  being 
used.  This  changes  the  time  history  of  the  upcoming  loop  input 
signal,  preventing  the  subject  from  learing  the  input  (recall 
section  A. 2. 1.3).  OCMRUN  also  provides  the  user  with  the  option  of 
recalibrating  the  stick  and  display  at  this  point  (see  section 
A. 2. 3 for  further  discussion). 

The  options  just  mentioned  provide  the  user  with  sufficient 
flexibility  for  changing  the  task  parameters  from  run  to  run,  if 
desired,  and  yet  provide  a relatively  streamlined  procedure  for 
conducting  sequential  replications  of  the  same  tracking  task. 

A. 2. 2. 5 Premature  Run  Termination 

Normally,  the  duration  of  a run  is  determined  by  the 
user-specified  run  length  Tp ; however,  certain  run-time  conditions 
can  cause  a premature  termination  and  a shortened  effective  run 
length . 

Prior  to  sampling  the  stick  signal,  OCMRUN  checks  that  the 
A-to-D  buffer  has  been  previously  read  from,  before  using  the 
buffer  to  store  the  newly  sampled  stick  voltage.  Under  unusual 
circumstances,  the  buffer  may  not  have  been  read  from  following  an 
earlier  A-to-D  conversion,  and  in  this  situation  OCMRUN  prompts  the 
user  with  an  appropriate  error  message.  A normal  run  termination 
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is  then  initiated,  as  described  in  section  A. 2. 2. 3,  resulting  in 
tracking  data  which  is  saved,  but  which  is  associated  with  a 
shorter  than  normal  tracking  run. 

During  real-time  control  of  the  tracking  run,  OCMRUN  also 
checks  for  an  abnormally  large  display  error  signal,  a condition 
which  indicates  that  the  subject  has  lost  control  of  the  task. 
Rather  than  continuing  the  task  under  these  circumstances,  OCMRUN 
prompts  the  user  with  an  appropriate  error  message  and  initiates 
run  termination  according  to  the  procedure  described  in  section 
A. 2. 2. 3.  Again,  a shorter  than  normal  run  is  the  result. 

OCMRUN  also  provides  a check  to  ensure  that  the  real-time 
signal  processing  is  completed  within  the  allotted  sample  interval. 
If  it  is  not,  OCMRUN  avoids  the  problem  of  desynchronization  by 
prompting  the  user  with  an  appropriate  error  message,  and 
Initiating  run  termination  as  in  the  previous  two  cases. 

In  the  three  cases  just  described,  the  tracking  data  is  saved 
ud  to  the  point  of  the  detected  error  condition.  This  is  not  the 
case,  however,  when  the  user  initiates  an  early  termination  from 
the  terminal,  by  typing  in  a'C.  In  this  case,  neither  the  task 
parameter  values  nor  the  data  are  saved  on  the  disc  file;  however, 
the  task  parameter  values  are  saved  in  computer  memory,  and  the 
user  can  restart  a run  using  normal  system  procedures  for 
restarting  program  execution. 

A. 2. 3 Cal ibr at  ion 

To  implement  the  tracking  loop,  provisions  must  be  made  for 
the  conversion  between  the  internal  digital  representation  of  a 
loop  variable  and  the  corresponding  external  analog  voltage.  This 
is  accomplished  by  the  use  of  a set  of  scale  factors  whose  values 
are  specified  during  a semi-automated  calibration  procedure. 
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Following  the  user  specification  of  the  task  parameter  values 
and  prior  to  the  start  of  a run,  OCMRUN  checks  for  the  presence  of 
a valid  set  of  scale  factors.  If  such  a set  is  found,  the  user  is 
provided  the  opportunity  to  generate  a new  set  by  recalibrating;  if 
the  user  decides  not  to  recalibrate,  OCMRUN  proceeds  to  the 
initialization  portion  of  the  run.  If  the  user  decides  to 
recalibrate,  or  if  a valid  set  of  scale  factors  is  not  present, 
OCMRUN  initiates  a calibration  procedure. 

OCMRUN  first  prompts  the  user  as  to  the  desired  maximum 
display  deflection,  in  centimeters.  To  take  full  advantage  of  the 
maximum  D-to-A  voltage  range,  OCMRUN  then  provides  minimum,  maximum 
and  zero  voltage  signals  while  prompting  the  user  to  adjust  the 
displav  scope  gain  and  zero  appropriately.  In  this  manner,  the 
user  can  quickly  calibrate  the  display  scope,  so  that  desired 
maximum  display  deflection  range  corresponds  to  the  maximum  D-to-A 
range.  At  the  same  time,  OCMRUN  calculates  the  output  scale  factor 
which  will  be  used  later  in  converting  from  the  digital  display 
error  (in  cm)  to  the  proper  D-to-A  signal  level  (in  volts). 

OCMRUN  also  prompts  the  user  for  two  stick  force  setpoints  to 
be  used  in  calibrating  the  stick,  A calibrated  stick  force  is 
provided  by  the  user  in  both  directions,  and  OCMRUN  obtains  two 
correspond ing  digital  values  from  A-to-D  sampling.  Using 
calibrating  setpoint  values  (in  pounds)  provided  by  the  user, 
OCMRUN  calculates  the  input  scale  factor  which  will  be  used  later 
in  converting  from  the  stick  signal  level  (in  volts)  to  the 
internal  digital  value  (in  pounds).  OCMRUN  also  calculates  and 
displays  the  maximum  allowable  stick  force  in  pounds,  along  with 
the  stick  bias  due  to  possible  asymmetry  in  the  set-points.  The 
user  is  then  provided  the  option  of  recalibrating  if  any  of  these 
values  are  unacceptable. 
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A. 3.  PERFORMANCE  ANALYSIS:  OCMANL 

This  section  provides  a closer  look  at  how  the  Performance 
Analyzer  analyzes  the  stored  data  obtained  during  a tracking  run. 
The  analysis  is  entirely  digital  and  is  implemented  by  the  use  of 
the  program  OCMANL.  The  program  provides  several  major  functions, 
ranging  from  data  file  reading  to  spectral  analysis,  and  like 
OCMRUN,  is  programmed  to  provide  an  interactive  dialog  with  the 
user.  The  analysis  functions  are  organized  into  eight  program 
units  which  are  individually  selectable  by  the  user;  these  units 
are  discussed  in  more  detail  in  the  following  subsections. 

A. 3.1  File  Reading 

The  user  will  normally  begin  the  data  analysis  by  reading  in 
the  appropriate  data  file  from  the  system  disc.  OCMANL  prompts  the 
user  for  the  data  file  name,  and  after  reading  the  header 
information  on  the  file,  prints  out  the  file's  name  and  the  run 
number,  date,  and  time  associated  with  the  file's  creation.  OCMANL 
then  displays  to  the  user  the  starting  point  used  for  generating 
the  scores  during  real-time  control  of  the  tracking  task.  The  user 
is  given  the  opportunity  of  changing  this  start  point,  since  this 
point  determines  which  portion  of  the  tracking  histories  will 
constitute  the  data  base  for  subsequent  analysis.  Finally,  OCMANL 
questions  the  user  as  to  his  choice  of  stick  units:  either  the 
recorded  units  of  pounds  force,  or  the  equivalent  plant  command 
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units  of  cm,  cm/sec,  or  cm/sec  , the  latter  being  calculated  from 
the  recorded  units  and  the  stored  stick  gain. 

A. 3. 2 Listing  of  Tracking  Task  Parameters 

Since  the  tracking  task  parameter  values  used  to  define  a run 
are  stored  with  the  data  of  that  run,  the  parameter  values  are 
available  for  inspection  by  the  user.  OCMANL  can  provide  a 
formatted  labelled  listing  of  all  the  parameter  values  of  interest, 
upon  request  from  the  user.  This  serves  as  a cross-check  of  the 
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actual  conditions  under  which  the  run  was  conducted,  and  thereby 
serves  as  a backup  to  a conventional  laboratory  notebook  or  log. 

A. 3. 3 Tracking  Scores 

On  request  from  the  user,  OCMANL  will  read  in  the  time 
histories  of  the  four  loop  variables  stored  on  the  data  file  (n*, 
s* , p* , and  e* ) , and  calculate  and  display  a mean,  standard 
deviation,  and  RMS  value  for  each  variable.  These  scores  are 
calculated  over  a time  interval  equal  to  the  period  Tq  of  the 
pseudo-random  loop  input  signal  n*,  with  scoring  beginning  at  the 
user  specified  start  point.  The  stick  score  is  calculated  in 
either  pounds  force  or  equivalent  display  units  (cm),  depending  on 
the  user's  earlier  choice. 

OCMANL  also  provides  the  user  with  the  option  of  calculating 
and  displaying  the  rate  scores  for  the  four  loop  variables.  The 
rate  scores  are  calculated  over  the  same  interval  and  in  the  same 
units  as  are  the  position  scores  just  described.  To  avoid  problems 
associated  with  numerical  differentiation,  a frequency  domain 
approach  is  used,  based  on  the  signal's  power  spectrum,  in  turn 
obtained  from  the  frequency  spectrum  of  that  signal  (see  next 
section)  . 

OCMANL  also  displays  to  the  user  the  overall  tracking  score, 
based  on  a weighted  sum  of  the  display  error  variance  and  stick 
signal  variance  (recall  equation  (A.2.7)). 

A. 3. 4 Signal  Spectra 

On  request  form  the  user,  OCMANL  will  read  in  the  time  history 
of  a Iood  variable,  perform  a fast  Fourier  transform  on  the 
digitized  data  contained  in  the  scoring  interval,  and  calculate  the 
signal's  associated  power  spectrum.*  Rather  than  display  the 

* Assuming  this  has  not  been  done  earlier  in  response  to  a user 
request  for  a rate  score  or  describing  function  calculation 
involving  that  signal  (see  sections  A. 3. 3 and  A. 3.5) 
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entire  spectrum,  OCMANL  diSDlavs  a more  compact  data  set, 
consisting  of  the  correlated  power  and  estimated  remnant  power  at 
the  correlated  frequencies.  These  frequencies  are  simply  those 
contained  in  the  loop  input  signal  (recall  the  of  section 
A. 2. 1.2).  The  correlated  power  is  thus  the  signal's  power  at  these 
frequencies . 

The  estimated  remnant  power  at  each  correlated  frequency  is 
obtained  bv  averaging  the  power  in  the  neighborhood  of  each 
correlated  frequency,  taking  care  to  exclude  any  correlated  power. 
The  size  of  this  neighborhood  is  set  at  a fixed  fraction  of  an 
octave,  resulting  in  a greater  number  of  remnant  power  measurements 
being  averaged  at  the  higher  frequencies.  OCMANL  displays  to  the 
user  the  number  of  measurements  comprising  each  average  at  each 
frequency . 


Using  these  signal  power  estimates,  OCMANL  also  estimates  the 
power  spectral  density  (PSD)  of  both  the  correlated  and  remnant 
components  of  the  signal.  This  facilitates  direct  comparisons  with 
human  operator  model  predictions,  since  models  typically  utilize 
remnant  sources  characterized  by  PSD's  which  are  continuous  with 
frequency . 

To  provide  the  user  with  an  indication  of  how  well-correlated 
the  signal  is  with  the  loop  input  signal,  OCMANL  calculates  and 
displays  the  correl ated- to-remnant  power  ratio  (C/R),  for  each 
frequency  in  the  loop  input  signal.  This  C/R  ratio  is  calculated 
for  both  the  signal  power  and  its  power  spectral  density,  the 
former  for  assessing  degree  of  correlation,  and  the  latter  for 
direct  comparison  with  typical  human  operator  model  predictions. 
Total  power  and  power  ratios  are  also  calculated  and  displayed; 
these  provide  the  user  with  an  indication  of  signal  coherence. 

114 


A. 3.5  Describing  Functions 

Tf  the  user  wishes  to  calculate  the  describing  function 
relating  any  one  loop  variable  to  any  other,  OCMANL  will  prompt  the 
user  for  the  appropriate  alphanumeric  symbol  identifying  each  of 
the  two  variables.  One  symbol  is  then  associated  with  the 
numerator  of  the  desired  describing  function  and  the  other  with  the 
denominator.  For  each  such  specified  variable,  OCMANL  reads  in  its 
time  history,  performs  a fast  Fourier  transform  on  the  digitized 
data  contained  in  the  scoring  interval  and  calculates  the  signal’s 
amplitude  and  phase  as  a function  of  frequency.*  The  amplitude 
ratio  (in  dB)  and  phase  difference  (in  degrees)  are  then  calculated 
at  each  loop  input  frequency,  and  displayed  to  the  user  in  tabular 
form . 

OCMANL  provides  two  other  functions  when  the  user  requests  a 
describing  function  calculation.  To  ensure  a smooth  progression  of 
phase  with  frequency,  OCMANL  checks  the  calculated  phase  at  each 
test  frequency.  If  found  to  be  within  360  degrees  of  the  phase 
calculated  at  the  preceding  test  frequency,  no  corrections  are 
made;  a larger  phase  difference,  however,  results  in  the  addition 
or  subtraction  of  360  degrees  as  necessary  to  bring  the  calculated 
phase  value  in  the  "neighborhood”  of  the  preceding  phase  value. 

The  second  function  provided  by  OCMANL  concerns  the  validity 
of  the  calculated  gains  and  phases  at  each  frequency.  Since  the 
describing  function  calculation  presumes  a negligible  remnant 
contribution  to  the  signal  power  at  the  loop  input  frequencies,  a 
significant  remnant  contribution  can  render  the  gain  and  phase 
calculations  invalid.  OCMANL  provides  the  user  with  a check  on 
remnant  power  by  utilizing  the  correl ated- to-remnant  power  ratio 
(C/R)  described  earlier  in  the  previous  section.  If  the  C/R  ratio 

* Assuming  this  has  not  been  done  earlier  in  response  to  a user’s 
request  for  a printout  of  the  signal's  rate  score  or  power 
spectrum  (see  sections  A. 3. 3 and  A.3.*0. 
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at  a particular  frequency  is  small,  indicating  a large  remnant 
contribution,  the  user  is  alerted  to  this  situation  by  OCMANL,  via 
a visual  flag  presented  with  the  tabulated  data  item. 

If  the  user  requests  the  calculation  of  the  human  operator 
describing  function  (stick/display).  OCMANL  provides  one  further 
function  for  the  user:  a change  of  sign.  Since  the  basic  tracking 
loop  of  figure  A.1  involves  a positive  feedback,  the  human  operator 
must  provide  a sign  reversal  to  ensure  loop  stability.  To 
eliminate  this  sign  reversal  contained  in  the  digitized  stick 
history,  and  ensure  consistency  with  conventional  human  operator 
measurements  made  with  negative  feedback  loops,  OCMANL  adds  180 
degrees  to  the  calculated  phases,  before  displaying  the  results  to 
the  user.  Again,  it  should  be  noted  that  this  occurs  only  when  the 
user  requests  the  calculation  of  the  human  operator  describing 
function;  other  describing  function  calculations  suffer  no  such 
adjustments . 

A. 3. 6 Observation  Noise 

On  request  from  the  user,  OCMANL  will  calculate  and  display  as 
a function  of  frequency  the  human  operator's  observation  noise, 
referred  to  both  display  and  display  rate.  The  noise  calculations 
are  based  on  the  approach  discussed  by  Levison  et  al  (1969),  and 
involve  the  stick  power  spectrum,  scores  for  display  and  display 
error  rate,  and  the  loop  input  power  referred  to  the  display.  The 
stick  power  and  the  scores  are  calculated  as  described  previously 
in  sections  A. 3.4  and  A. 3. 3.  respectively;  the  input  power  is 
calculated  analytically,  based  on  the  input  signal  parameters  and 
the  plant  dynamics.  OCMANL  ensure  that  these  calculations  are 
sequenced  properly  and  displays  the  calculated  observation  noise  at 
each  loop  input  frequency  in  an  appropriate  tabular  format. 

An  additional  function  provided  by  OCMANL  concerns  the 
validity  of  the  calculated  observation  noise  at  each  loop  input 
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frequency.  Since  remnant  power  estimates  may  not  exist  at  low 
frequencies  (due  to  the  user's  choice  of  run  length),  OCMANL  checks 
for  their  presence.  If  an  estimate  is  found  missing,  no  noise 
calculation  is  made  at  that  frequency  and  the  user  is  alerted  to 
this  situation,  via  a visual  flag  presented  with  the  tabulated  data 
item.  In  addition,  OCMANL  provides  for  such  flagging  when  a low 
stick  C/R  power  ratio  is  associated  with  a particular  test 
frequency  (recall  section  A . 3 . *♦ ) , since  a low  ratio  may  invalidate 
the  associated  observation  noise  estimate. 

A. 3. 7 Amplitude  Distribution 

A. 3.8  Summary  Calculations  and  File  Writing 

On  request,  OCMANL  will  provide  the  user  with  a concise 
summary  of  the  fundamental  human  operator  measurements  made 
possible  by  the  use  of  the  Performance  Analyzer.  This  user  option 
performs  the  data  file  reading  (see  section  A. 3.1),  calculation  of 
the  tracking  scores  (see  section  A. 3.3)  for  the  four  loop  variables 
(loop  input,  stick,  plant,  and  display)  and  for  one  of  the  rate 
variables  (stick  rate).  In  addition,  the  human  operator's 
describing  function  is  calculated  (see  section  A. 3. 5),  as  is  his 
estimated  observation  noise  (see  section  A. 3.6).  As  described 
previously,  the  describing  function  and  noise  figures  are  flagged 
appropriately  to  alert  the  user  to  marginally  valid  or  invalid 
data . 

OCMANL  provides  the  user  with  the  options  of  displaying  the 
calculations  on  the  terminal  and/or  writing  the  results  on  a disc 
file.  If  the  user  chooses  the  latter  option,  OCMANL  prompts  the 
user  for  an  appropriate  file  name,  writes  the  data  on  the  file,  and 
closes  the  file  to  preserve  it  for  later  use.  This  allows  the  user 
to  condense  large  volumes  of  tracking  time  histories  into  small 
files  of  essential  frequency  domain  information,  which  can  be  later 
processed  in  an  ensemble  fashion  to  obtain  statistical  descriptions 
of  tracking  behavior. 
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A. *4.  PERFORMANCE  ANALYZER  OPERATION 

This  section  provides  examples  of  the  Performance  Analyzer 
capabilities  just  described,  and  illustrates  user  interaction 
during  problem  definition  and  tracking  analysis.  For  convenience, 
the  description  is  divided  between  the  two  operating  programs, 
OCMRUN  and  OCMANL . 

A. *4.1  Tracking  Task  Implementation:  OCMRUN 

OCMRUN  provides  the  user  with  the  capability  of  defining  the 
tracking  task  parameters,  controlling  the  tracking  task  itself,  and 
calibrating  the  interface  modules.  The  following  subsections 
illustrate  these  functions  in  more  detail. 

A. 4. 1.1  Definition  of  the  Tracking  Task  Parameters 

The  listing  given  on  the  following  pages  illustrates  how  the 
user  can  define  the  desired  tracking  task  by  responding  to 
appropriate  program  prompts.  User  responses  are  circled  and 
labelled  . 

The  user  initiates  program  execution  (A),  and  indicates  that 
the  parameters  will  be  entered  from  the  terminal  rather  than  read 
from  a parameter  file  (B).  The  user  is  then  prompted  for  the 
sample  period  (C)  and  run  time  (D)  and  the  program  lists  the  run 
length  and  input  signal  period  .both  in  seconds  and  in  number  of 
sample  points,  as  a check  to  the  user.  The  input  signal  period,  in 
terms  of  the  number  of  sample  points,  is  chosen  as  the  largest 
power  of  two  which  is  less  than  the  run  length,  in  sample  points; 
in  this  case,  the  input  signal  period  is  *4096.  The  program  then 
checks  with  the  user  if  this  is  satisfactory,  and  the  user  responds 
affirmatively  (E).  If  the  choice  had  not  been  satisfactory,  the 
user  would  have  been  provided  the  opportunity  of  reentering  a 
different  sample  period  and/or  run  length. 
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f ocmrunJ^ 


INITIALIZING  PARAMETERS  FROM  A FILE 

ItltlT I ME  BASE  PARAMETERS***** 
SAMPLE  PERIOD  IN  MILL  I SEC 
RUN  TIME  IN  SECONDS  :<18^ — D 


<*T 


B 


RUN  LENGTH* 

INPT  SIGNAL  PERIOD* 


185.00  SECONDS  AND  4626  POINTS 
163.84- SECONDS  UITH  4096  POINTS 


■0-E 


0K?( 

SCORING  START  TIME* 

ok*(y) F 

UANT  TIME  BASE  PARAMETERS  LISTED?! 


10.60  SECONDS  AT  POINT  266 

G 


er 


SAMPLE  PERIOD (SECONDS)  * 4.000E-02 

NUMBER  OF  POINTS  IN  RUN=  4626 

RUN  DURATION ( SECONDS ) = 1.850E+02 

***** I NPUT  SIGNAL  PARAMETERS**#** 

USING  AN  EXTERNAL  FORCING  FUNC  T ION?(jT)r^ 

NHAT  NIND<  T OR  D)?^) 1 

PSD  FUNCTION  TYPE(1  OR  21(77) J 

POLE  location:^?) — K 

RMS  LEVEL 

UANT  THE  NOMINAL  SET  OF  FREQUENCIES?! 


H 


UANT  THE  FREQUENCIES  LISTED?! 


€r 


M 


N 


COMP 

HARM 

UACT 

UDES 

1 

7 

0.27 

0.25 

2 

13 

0.50 

0.50 

3 

29 

1.11 

1.00 

4 

37 

1.42 

1.41 

5 

53 

2.03 

2.00 

6 

73 

2.80 

2.83 

7 

103 

3.95 

4.00 

8 

149 

5.71 

5.66 

9 

211 

8.09 

8.00 

10 

293 

11.24 

11.30 

11 

41? 

16.07 

1 6'.  00 

12 

587 

22.51 

22.60 

13 

839 

32.  10 

32.00 

OK  ?l 


G>° 
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WANT  THE  NOMINAL  SET  OF  PHASES 


WANT  THE  PHASES  LISTED 


t(0—  Q 


■0- 


COMP 

PMULT 

PHASE 

1 

2019 

177.45 

2 

3865 

339.70 

3 

919 

80.77 

4 

3494 

307.09 

5 

408 

35.86 

6 

3765 

330.91 

7 

2534 

222.71 

S 

1794 

157.68 

9 

247 

21.71 

10 

1711 

150.38 

11 

3949 

347.08 

12 

103 

9.05 

13 

1933 

169.89 

WANT  THE  INPUT  SIGNAL  PARAMS  LISTED 


TASK  IS  DISTURBANCE  NULLING 

NUMBER  OF  POINTS  IN  INPUT  SIGNAL=  4096 
INPUT  SIGNAL  DURATION ( SECONDS ) = 1.638E+02 

BASE  FREQUENCY < RAD/S)  = 3.835E-02 

PSD  FUNCTION  IS  FIRST  ORDER 


POLE  LOCATION<  RAD/SEC ) = 
RMS  INPUT  LEVEL 


5.000E-01 
1 .OOOE+OO 


SPECTRAL  COMPOSITION  OF  INPUT  SIGNAL 


IMP 

HARM 

FREQ 

AMP 

PHASE 

1 

7 

0.27 

0.90 

177.45 

2 

13 

0.50 

0.67 

339.70 

3 

29 

1.11 

0.52 

80.77 

4 

37 

1.42 

0.34 

307.09 

5 

53 

2.03 

0.32 

35.86 

6 

73 

2.80 

0.27 

330.91 

7 

103 

3.95 

0.24 

222.71 

8 

149 

5.71 

0.20 

157.68 

9 

211 

8.09 

0.16 

21.71 

10 

293 

11.24 

0.14 

150.38 

11 

419 

16.07 

0.12 

347.08 

12 

587 

22.51 

0.10 

9.05 

13 

839 

32.18 

0.08 

169.89 

*****PIANT  PARAME TERS***** 
ENTER  PLANT  TYPE < 0 THRU  6 >(jT)''' 


GAI H-ZJ—  T 
POLE  5(^2) — U 

TINE  DELAY  IN  M II  L ISFC ^25^""  ^ 

ACTUAL  DELAY  WILL  BE  20  MILIISEC.  OK'T^Y^T^ 

STICK  GAIN  IN  CM/LB^T)—  X 

WANT  PLANT  PARAMETERS  LISTED  •’(TT)"'  ^ 

PLANT  TYPE  ■ 4 

GAIN  * 2. OOOE+OO 

POLE  < RAD/S ) - -2 • OOOE'+OO 

TIME  DELAY  » 2.000E-02 


STICK  GAIN 


5. OOOE+OO 


*****SCORlNG  PARAMETERS***** 


STICK  WEIGHTING! 


(°)~ 


WANT  SCORING  PARAMETERS  LISTED 

STICK  WEIGHTING*  0.000E-01 
SCORING  STARTS  AT  POINT  246 
AND  STOPS  AT  POINT  4341 


*****CAL I BRAT  ION  PARAMETERS***** 

FIRST  CALIBRATION  VALUE(LR)  1^7^ — 

SECOND  CALIBRATION  VALUE<LB>(fl) — CC 

DD 

WANT  CALIBRATION  PARAMETERS  LISTED^Yj" 

FIRST  CALIBRATION  LEVEL  » -1. OOOE+OO 
SECOND  CALIBRATION  LEVEL-  1. OOOE+OO 


I 


WANT  ALU  PARAMETERS  LISTED  OUf^jT)—  EE 

******************** ************************************ ************** 

*****TIME  BASE  PARAMETERS***** 

SAMPLE  PER I OP (SECONDS)  * 4.00QE-02 

NUMBER  OF  POINTS  IN  RUN*  4626 

RUN  DURATION (SECONDS)  = 1.850E+02 

*****INPUT  SIGNAL  PARAMETERS***** 

TASK  IS  DISTURBANCE  NULLING 

NUMBER  OF  POINTS  IN  INPUT  SIGNAL*  4096 
INPUT  SIGNAL  DURATION (SECONDS)  •=  1.63HE+02 

BASE  FREQUENCY (RAD/S)  = 3.835E-02 

PSD  FUNCTION  IS  FIRST  ORDER 

POLE  l.OCAT  I ON  ( RAD/SEC  >»  5.000E-01 
RMS  INPUT  LEVEL  * l.OOOEPOO 

SPECTRAL  COMPOSITION  OF  INPUT  SIGNAL 


COMP 

HARM 

FREQ 

AMP 

PHASE 

1 

7 

0.27 

0.90 

177.45 

2 

13 

0.50 

0.67 

339.70 

3 

29 

1.11 

0.52 

80 . 77 

4 

37 

1.42 

0.34 

307.09 

5 

53 

2.03 

0.32 

35.86 

6 

73 

2.80 

0.27 

330.91 

7 

103 

3.95 

0.24 

222.71 

8 

14<> 

5.71 

0.20 

157.68 

9 

211 

8.09 

0.16 

21.71 

10 

293 

11.24 

0.14 

150.38 

11 

419 

16.07 

0.12 

347.08 

12 

587 

22.51 

0.10 

9.05 

13 

839 

32.18 

0.08 

169.89 

*****Pl.ANT  PARAMETERS***** 


PLANT  TYPE  * 4 
GAIN  * 2.000E+00 

POLE ( RAD/S) = -2.000E+00 

TIME  DELAY  « 2.000E-02 

STICK  GAIN  * S.OOOE+OO 

* * * * *SCOR  INC.  P AR  AME  TE  RS*  * **  * 

STICK  WEIGHTING--  0.0001  -Ot 
SCORING  STARTS  AT  POINT  266 
AND  STOPS  AT  POINT  4661 

♦ ♦♦♦•CALIBRATION  PARAMI  TERS***** 

FIRST  CALIBRATION  L l VI  I -1.0001  LOO 

SECOND  CALIBRATION  I F VI: I 1.000!  TOO 

*************'f*M**t'MT*****t****’M*********t  ************  ******  ******* 
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5’©" 


FF 


ANY  CHANGES 
CREATING  A PARAMETER  FILETftY 


ENTER  FILE  NAME  FOR  OUTPUT  JfTEMP.  PAR  ] 
UOING  A RUN  NOW  ''(ffT)—  1 1 
REINITIALIZING  PARAMETERS 
INITIALIZING  PARAMETERS  FROM  A FILE7! 


€r 


jj 


REINITIALIZING  PARAMETERS 


INITIALIZING  PARAMETERS  PROM  A FILE'»(jT)r’ 
ENTER  Flip  NAME  FOR  INPUT  IfTEMP . PAR  ' 

WANT  AIL  PARAMETERS  LISTED  OUT’/n*' 

ANY  CHANGE ST^Y^ — ^ 

WHICH  PAR T v < 0 THRU  5>^3 NN 

MIHP1  AN T PARAMETERS***** 

ENTER  PLANT  TYPE<0  THRU  A ) 

GAINJ&OCf- PP 
ZETATf.  707V-  QQ 
UN^_  RR 

TIME  DELAY  IN  MILLISEC 
STICK  GAIN  IN  CM/LI 
WANT  PLANT  PARAMETERS  LISTED’ 


‘€r 


OO 


■c@r 

10-  TT 


ss 


.0r 


uu 


PLANT  TYPE  <=  A 

GAIN  - 1 .000E+02 

ZETA  “ 7.070E-01 

UN  » l.OOOE+Ol 

TIME  DELAY  » O.OOOE-OI 


CiES  0- 


5 • OOOE+OO 

VV 


STICK  GAIN 
MOKE  CHANGES’ 

WANT  ALl  PARAMETERS  LISTED  OUT’ 
CREATING  A PARAMETER  TILERY  • XX 


ENTER  FILE  NAME  FOR  OUTPUT  SfTEMPl  .PAR 

zz 


DOING  A RUN  NOW’ 
REINITIALIZING  PARAMETERS 


J0- 


124 


JJ 


- - 


The  program  then  lists  the  time  at  which  scoring  will  begin, 
here  chosen  so  as  to  "center"  the  scoring  interval  of  163.84 
seconds  within  the  run  interval  of  185  seconds.  The  program  then 
checks  with  the  user  if  this  is  satisfactory  and  the  user  responds 
affirmatively  (F).  If  the  scoring  start  time  had  not  been 
satisfactory,  the  user  would  have  been  provided  the  opportunity  of 
entering  a different  start  time.  Finally,  the  program  lists  the 
time  base  parameters  in  response  to  the  user's  request  (G). 

The  user  is  then  prompted  for  information  concerning  the  input 
signal,  after  requesting  that  one  be  used  in  the  tracking  task  (H). 
The  user  then  specifies  a disturbance  input  (rather  than  a target 
input)  (I),  the  order  and  pole  location  of  the  power  spectral 
density  function  to  be  approximated  (J,K),  and  finally  the  desired 
RMS  value  of  the  input  signal  (L).  The  user  then  indicates  that  he 
wishes  to  use  the  nominal  set  of  pre-defined  input  signal 
frequencies  (M),  and  then  requests  a listing  of  them  (N).  Had  the 
user  not  wished  to  use  the  nominal  frequency  set,  the  program  would 
have  provided  him  the  opportunity  of  defining  his  own  set  of 
frequencies.  In  either  case,  the  affirmative  answer  to  the  listing 
request  (N)  results  in  a listing  as  shown:  the  frequency  component 
is  given  first  (COMP),  followed  by  the  harmonic  multiplier  (HARM), 
the  actual  input  frequency  in  radians/sec  (WACT),  and  the  desired 
frequency  (WDES)  . The  desired  frequencies  (WDES)  are  those 
specified  by  the  user  and,  in  this  case,  are  those  comprising  the 
nominal  frequency  set  having  half-octave  jumps.  The  actual 
frequencies  (WACT)  differ  from  the  desired  frequencies  since  the 
actual  frequencies  are  chosen  to  be  harmonics  of  a base  frequency 
and  the  harmonics  (HARM),  in  turn,  are  chosen  to  be  all  prime 
numbers  . 

After  listing  the  frequencies,  the  program  then  checks  with 
the  user  if  this  is  satisfactory  and  the  user  responds 
affirmatively  (0).  If  this  frequency  set  had  not  been 
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satisfactory,  the  user  would  have  been  provided  the  opportunity  of 
changing  either  the  entire  set  of  desired  fre-quencies  or  specific 
components  of  it.  The  user  would  have  also  been  given  the  choice 
of  relaxing  the  constraint  of  prime  harmonics  in  favor  of  simple 
integer  harmonics. 

The  user  then  indicates  that  he  wishes  to  use  the  nominal  set 
of  phases  associated  with  the  input  signal  sinusoids  (P),  and 
requests  a listing  of  them  (Q).  The  listing  shown  gives  the 
component  number  (COMP),  the  integer  phase  multiplier  (PMULT)  and 
the  phase  in  degrees  (PHASE).  For  each  component,  the  phase 
multiplier  is  chosen  to  be  a random  number  between  zero  and  the 
number  of  points  in  the  input  signal  (here,  4096);  and  the  phase 
itself  is  calculated  correspondingly  between  zero  and  360  degrees. 
The  resulting  string  of  random  phases  is  based  on  a random  number 
generator  "seed"  of  unity;  had  the  user  responded  negatively  to  the 
question  concerning  the  use  of  the  nominal  phase  set  (P),  the 
program  would  have  provided  him  with  the  opportunity  of  choosing  a 
different  "seed"  value  to  be  used  to  generate  a different  string  of 
random  phases. 

By  responding  affirmatively  to  an  input  signal  parameter 
listing  request  (R),  the  user  is  provided  with  a summary  of  the 
signal  parameters  as  shown.  In  addition  to  the  parameters  just 
described,  the  program  lists  the  amplitudes  (AMP)  associated  with 
each  component  sinusoid,  calculated  so  as  to  approximate  the 
specified  first-order  power  spectral  density  function  and  RMS 
1 evel . 

The  user  is  then  prompted  for  the  type  of  plant  dynamics  to  be 
incorporated  in  the  tracking  task.  Here  the  user  responds  (S)  with 
a 4,  indicating  a first-order  plant;  had  he  responded  with  a zero, 
a listing  of  the  plant  codes  would  have  been  provided.  The  user 
then  specifies  the  plant  gain  (T)  and  pole  location  ( U ) ; the 
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negative  value  corresponds  to  an  unstable  pole.  The  user  also 
specifies  a 25  millisecond  dead-time  to  be  included  in  the  plant 
dynamics  (V)  but  the  program  restricts  this  to  be  an  integral 
multiple  of  half  the  sample  period  (in  this  example,  20  msec).  The 
program  then  checks  with  the  user  if  this  choice  is  satisfactory 
and  the  user  responds  affirmatively  (W).  Had  this  not  been 
satisfactory,  the  user  would  have  been  provided  the  opportunity  of 
reentering  a different  value  for  the  time  delay. 

The  user  then  specifies  the  stick  gain  value  (X),  which  is 
used  to  convert  the  stick  force  in  pounds  to  centimeters  of  display 
units.  A single  or  double  integrator  in  the  plant  dynamics  would 
have  resulted  in  a prompt  for  a corresponding  velocity  or 
acceleration  conversion  factor.  The  affirmative  answer  to  the 
listing  request  (Y)  results  in  a summary  of  the  chosen  plant 
parameter  values. 

The  user  is  then  prompted  for  the  stick  weighting  factor  to  be 
used  in  calculating  the  overall  tracking  score.  The  zero  response 
by  the  user  (Z)  implies  that  the  tracking  score  is  to  be  calculated 
on  the  basis  of  display  displacement  alone.  The  affirmative 
response  to  the  listing  request  (AA)  displays  the  stick  weighting 
and  the  scoring  start  and  stop  points,  the  last  two  parameters 
having  been  defined  earlier  with  the  time  base  parameter  selection. 

The  user  is  then  prompted  for  the  two  values  to  be  used  in 
calibrating  the  stick  force.  The  user  indicates  that  one  pound 
force  levels  will  be  used  (BB,CC);  the  negative  sign  of  the  first 
value  indicates  a left  stick  deflection  and  the  positive  sign  of 
the  second  value  indicates  a right  stick  deflection.  The 
affirmative  response  to  the  listing  request  (DD)  results  in  a 
listing  of  these  two  values. 

At  this  point,  the  parameter  specifications  are  complete  and 
the  user  is  provided  the  opportunity  of  listing  the  entire 
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parameter  set  defining  the  tracking  task.  His  affirmative  answer 
(EE)  results  in  the  tracking  task  parameter  summary  shown.  At  the 
end  of  this  listing,  he  is  provided  the  opportunity  of  changing  any 
of  the  parameter  values  currently  specified.  Here,  he  responds 
negatively  (FF),  but  had  he  responded  affirmatively,  the  program 
would  have  provided  appropriate  prompts  to  effect  the  desired 
parameter  value  changes. 

The  user  is  then  questioned  as  to  the  creation  of  a parameter 
file.  The  affirmative  response  (GG)  results  in  a prompt  for  a file 
name,  which  the  user  then  provides  (HH).  After  a check  on  the 
legality  of  the  fil*  name,  the  program  opens  a disk  file  with  that 
name,  writes  all  of  the  tracking  task  parameter  values  into  the 
file,  and  then  closes  the  file.  This  ensures  that  the  parameter 
values  will  be  saved  for  future  use,  thus  obviating  the  need  for  a 
repetition  of  the  parameter  specification  procedure  just  described. 

With  the  parameter  values  specified  and  saved  at  this  point, 
the  user  is  provided  the  opportunity  of  conducting  a tracking  run. 
Had  the  user  responded  affirmatively  at  this  point  (II),  final 
initialization  of  the  run  parameters  would  have  been  conducted  (see 
next  section).  However,  the  negative  response  (II)  results  in  a 
reinitialization  message,  followed  by  a prompt  as  to  the  source  of 
the  new  parameter  set  (JJ).  At  this  point,  the  program  has  looped 
back  to  the  initial  question  asked  of  the  user  (B),  and  the 
parameter  definition  process  can  be  repeated. 

If  the  user  wishes  to  create  multiple  parameter  files  defining 
different  tracking  tasks  (e.g.,  different  plant  dynamics),  he  has 
two  choices:  a parameter-by-parameter  specification  as  just 
outlined,  executed  for  each  desired  parameter  file;  or  a change  of 
the  desired  parameters  from  one  set  to  another,  creating  a new  file 
with  each  new  set.  The  former  situation  is  illustrated  in  the 
previous  example,  where  the  user  enters  a second  pass  of  individual 
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specification  of  all  the  tracking  task  parameters,  by  answering 
negatively  to  the  last  program  prompt  (JJ). 

The  alternative  approach  to  multiple  parameter  file  creation 
is  illustrated  in  the  listing  below,  which  begins  with  the  user 
requesting  that  the  parameter  set  be  read  in  from  a previously 
created  file  (JJ).  He  is  then  prompted  for  the  parameter  file 
name,  and  after  giving  it  (KK)  , is  asked  if  he  wishes  a listing  of 
the  complete  parameter  set.  His  negative  response  (LL)  results  in 
a prompt  for  a change  request  and  his  affirmative  answer  here  (MM) 
results  in  a request  for  a specification  of  the  parameter  subset  to 
be  changed.  Here,  the  user  responds  (NN)  with  a 3,  indicating  a 
change  in  the  plant  parameter  set;  had  he  responded  with  a zero,  a 
listing  of  the  parameter  set  codes  would  have  been  provided. 

At  this  point  the  user  can  specify  a new  set  of  plant 
parameters,  differing  from  the  ones  given  in  the  previous  example. 
Here,  he  specifies  that  the  dynamics  be  second-order  (00),  and 
specifies  the  gain  (PP),  damping  ratio  (QQ),  and  natural  frequency 
(RR).  No  time  delay  is  desired  so  the  user  responds  with  a zero 
(SS).  After  specifying  the  stick  gain  (TT),  the  user  has  the  plant 
parameters  listed  (UU) . 

Since  the  user  wishes  to  change  only  the  plant  dynamics,  he 
indicates  that  no  further  changes  are  to  be  made  (VV),  and  that  a 
complete  parameter  set  listing  is  unnecessary  (WW).  However,  since 
the  user  does  wish  this  new  parameter  set  to  be  saved,  he  responds 
affirmatively  to  the  parameter  file  query  (XX),  and  provides  a new 
file  name  for  parameter  storage  (YY).  A negative  answer  to  the  run 
initialization  question  (ZZ)  results  in  a program  loop  back  to 
parameter  reinitialization  (JJ). 

With  the  procedure  Just  described,  the  user  can  rapidly 
create  several  parameter  files  for  different  tracking  situations  by 
changing  only  a few  parameters  at  a time. 
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A.U.1.2  Run  Identification  and  File  Labelling 

The  listing  given  on  the  following  page  illustrates  how  the 
user  can  initiate  a run  once  the  tracking  task  parameters  have  been 
defined . 

The  user  initiates  program  execution  (A),  and  indicates  that 
the  parameters  are  to  be  read  from  a previously-created  parameter 
file  (B).  He  is  then  prompted  for  the  parameter  file  name  and 
after  giving  it  (C).  is  asked  if  he  wishes  a listing  of  the 
complete  parameter  set.  His  negative  response  (D)  results  in  a 
prompt  for  a change  request  to  which  he  also  responds  negatively 
(E). 

Since  the  user  indicates  that  no  parameter  file  is  to  be 
created  (F),  the  program  assumes  that  the  user  wishes  to  conduct  a 
tracking  run  with  the  current  parameter  set.  Before  continuing, 
however,  the  program  first  checks  for  the  presence  of  valid  stick 
and  display  input/output  scale  factors,  factors  calculated  on  the 
basis  of  a previously-conducted  calibration  (see  section  A. 4.  1.4). 
In  this  example,  a calibration  had  been  conducted  earlier  and  the 
results  stored  in  the  specified  input  file  TEMP.DAT;  thus,  the 
program  questions  the  user  as  to  whether  or  not  he  wishes  to  use 
this  old  set  of  scale  factors.  The  user's  response  is  affirmative 
(G),  thus  bypassing  a calibration.  If  the  user's  response  had  been 
negative,  or  if  the  input  file  had  not  contained  a valid  set  of 
scale  factors,  the  user  would  have  been  forced  to  conduct  a 
calibration  to  generate  a »alid  set. 

The  program  then  prints  out  the  run  number  and  current  date. 
In  this  case  the  run  number  is  unity  because  this  is  the  first  run 
following  program  execution  (A);  repeated  runs  conducted  by  the 
program  result  in  an  automatic  incrementing  of  the  run  number.  In 
either  case,  the  user  is  given  the  opportunity  of  changing  the  run 
number;  here,  he  chooses  not  to  (H)  . 
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The  user  is  then  prompted  for  the  number  of  lines  of  comment 
he  wishes  to  enter.  His  non-zero  response  (I)  results  in  an 
exclamation  mark  prompt  character  for  each  line  of  input;  the  user 
input  (J,K)  is  then  stored  with  the  other  parameters  defining  the 
tracking  task.  Had  the  user  specified  zero  lines  of  comment,  no 
prompts  would  have  been  provided  and  no  comments  expected  by  the 
program . 

The  user  is  then  prompted  for  the  file  name  to  be  used  for 
storage  of  the  tracking  task  parameters  and  the  tracking  data.  The 
user  response  (L)  is  then  checked  for  legality  and  if  valid,  a disc 
file  of  that  name  is  opened  and  used  to  store  the  tracking 
parameters.  The  file  remains  open  and  ready  to  receive  the 
tracking  data  as  it  is  generated. 

Finally,  the  user  is  prompted  for  a start  signal  (M)  which 
initiates  control  of  the  tracking  task.  Under  normal 
circumstances,  control  does  not  return  to  the  user  until  the  end  of 
the  specified  run  length,  although  certain  conditions  can  result  in 
early  run  termination  (see  below). 

A. 4.  1.3  Run  Termination  and  Multiple  Runs 

The  listing  on  the  following  page  illustrates  program  output 
at  the  end  of  a run  and  demonstrates  how  the  user  can  initiate 
additional  runs  with  a minimum  of  terminal  entries. 

The  user  starts  the  run  by  responding  with  the  start  character 
(A).  The  tracking  run  is  then  conducted  under  control  of  the 
program  until  the  designated  run  time  has  elapsed.  At  this  point 
the  tracking  data  is  saved  on  the  user-specified  disc  file  (in  this 
case,  A1.TMV)  and  summary  scoring  information  is  presented  on  the 
terminal  as  shown.  Mean,  standard  deviation,  and  RMS  values  are 
given  for  the  input  signal  (INPT),  stick  deflection  (STCK),  plant 
(PLNT),  and  display  displacement  (DISP).  In  addition,  the 
percentage  of  overloads  (OLPCT)  occurring  for  the  stick  and  display 
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TYPE  S TO  start:(sV-  a 
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S.D. 
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0.54 
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0.54 

0.54 

0.00 
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0.54 
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0 
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are  given,  along  with  the  points  at  which  these  initially  occurred 
during  the  run  (OLPNT);  in  this  example,  no  overloads  were  present. 
Finally,  the  overall  tracking  score  (COST)  is  given  based  on  the 
stick  and  display  scores  and  the  stick  weighting  factor  (which  is 
zero  in  this  case) . 

The  user  is  then  questioned  as  to  whether  or  not  he  wishes  to 
conduct  another  tracking  run.  A negative  response  here  would  have 
terminated  program  execution.  The  affirmative  response  (B), 
however,  results  in  a change  request  prompt.  Had  the  user 
requested  a change,  he  would  have  been  provided  the  opportunity  of 
changing  the  entire  parameter  set  (by  reading  in  a new  parameter 
file)  or  of  changing  only  a few  parameters  of  interest.  The  user’s 
negative  response  (C),  however,  results  in  an  incrementing  of  the 
run  number,  followed  by  a series  of  program  prompts  and  user 
responses  (D,E,F,G)  which  initiate  another  tracking  run.  In  this 
example,  the  user  elects  to  conduct  a third  tracking  run  (H),  once 
the  tracking  scores  from  the  second  have  been  displayed. 

Early  termination  of  a tracking  run  can  occur  if  : a)  the 
subject  loses  control  of  the  task;  b)  an  A-to-D  conversion  error 
occurs;  or  c)  a timing  error  occurs.  Should  one  of  these 
situations  occur,  an  appropriate  error  message  is  displayed  on  the 
terminal  at  the  of  occurrence,  the  run  is  halted,  and  the  tracking 
scores  are  displayed  in  the  manner  just  illustrated.  The  user  is 
then  provided  the  opportunity  of  conducting  another  run,  if 
desired . 

A. 4. 1.4  Calibration 

The  listing  given  on  the  following  page  illustrates  how  the 
user  can  calibrate  the  stick  and  display  and  generate  a valid  set 
of  input/output  scale  factors. 
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The  user  initiates  program  execution  (A),  and  indicates  that 
the  parameters  are  to  be  read  from  a previously-created  parameter 
file  (B).  He  is  then  prompted  for  the  parameter  file  name  and 
after  giving  it  (C),  declines  to  list  out  the  parameters  (D),  make 
changes  (E),  or  create  a new  parameter  file(F).  At  this  point,  the 
program  checks  for  the  presence  of  a valid  set  of  input/output 
scale  factors  and,  finding  none,  indicates  to  the  user  that  a 
calibration  is  required. 

The  user  is  then  prompted  for  the  maximum  deflection  of  the 
display  scope  expressed  in  centimeters.  After  the  user  response 
(G),  he  is  prompted  to  zero  the  display  bar  present  on  the  scope, 
using  the  scope  centering  controls.  Once  he  has  completed  this,  he 
enters  a carriage  return  (H),  and  is  then  prompted  to  adjust  the 
full  scale  (left)  scope  deflection  to  the  specified  maximum  (in 
this  case,  5 cm).  A second  carriage  return  (I)  results  in  a 
similar  prompt  for  adjustment  of  the  right  maximum,  and  the  third 
carriage  return  (J)  results  in  a prompt  by  the  program  checking  for 
satisfactory  scope  adjustment.  The  user’s  affirmative  response  (K) 
indicates  a satisfactory  adjustment.  Had  the  user's  response  been 
negative,  the  scope  adjustment  procedure  would  have  been  repeated, 
thus  allowing  the  user  to  readjust  the  scope's  centering  and  gain 
controls  . 

Once  the  display  calibration  is  complete,  the  user  can  then 
proceed  to  calibrate  the  stick.  The  user  is  prompted  to  set  the 
stick  at  the  first  calibration  level  specified  by  the  calibration 
parameter  set  (recall  section  A.^.1.1)  In  this  case,  the  parameter 
set  calls  for  a calibration  level  of  -1,  correspond ing  to  a one 
pound  force  applied  to  the  stick  in  the  leftward  direction.  The 
user  is  expected  to  apply  this  force  to  the  stick,  enter  a carriage 
return  (L),  and  hold  this  force  for  approximately  5 seconds. 
During  this  time  the  program  samples  the  stick  voltage  and 
calculates  an  average  value  over  the  sampling  interval,  a value 
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which  is  used  later  to  generate  a stick  force-to-vol tage  scale 
factor.  The  user  is  then  prompted  to  apply  the  second  calibration 
force  level  to  the  stick  and  a similar  sampling  procedure  is 
initiated  with  another  carriage  return  (M). 

With  the  two  force-voltage  points  thus  obtained,  the  program 
calculates  the  force-to-vol tage  scale  factor.  Using  this  and  the 
maximum  range  of  the  A-to-D  converter,  the  program  also  calculates 
the  corresponding  maximum  and  minimum  stick  forces;  the  values 
shown  on  the  listing  indicate  these  saturation  levels  for  the  stick 
force  input.  The  program  also  calculates  the  equivalent  stick 
force  bias  (ZERO  STICK),  which  is  the  force  level  corresponding  to 
zero  stick  voltage.  This  bias  is  then  expressed  in  terms  of 
full-scale  stick  range  to  indicate  to  the  user  the  degree  of 
left-right  asymmetry  in  the  stick.  Finally,  the  program  checks  if 
the  user  is  satisfied  with  this  calibration.  The  user’s 
affirmative  response  (N)  indicates  that  such  is  the  case.  Had  the 
user's  response  been  negative,  the  stick  calibration  procedure 
would  have  been  repeated. 

Before  proceeding  with  the  tracking  run  initialization,  the 
newly  created  input/output  scale  factors  are  stored  with  the 
tracking  task  parameter  set.  Thus,  the  scale  factors  are  saved 
with  the  parameters  when  a data  file  is  created,  making  it  possible 
for  the  user  to  conduct  later  runs  without  the  need  for  a new 
cal ibration . 

A .4.2  Performance  Analysis:  OCMANL 

OCMANL  provides  the  user  with  the  capability  of  flexible 
analysis  of  the  stored  tracking  data,  including  score  calculations, 
spectral  analysis,  and  describing  function  computations.  The 
following  subsections  illustrate  these  functions  in  more  detail. 
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A. 4. 2.1  File  Reading 


The  listing  given  on  the  following  page  illustrates  how  the 
user  begins  the  data  analysis  by  reading  the  appropriate  disc  file. 

The  user  initiates  program  execution  (A)  and  indicates  that  he 
is  unfamiliar  with  the  executive  structure  of  the  program  by 
entering  a zero  in  response  to  the  part  query  ( B) . This  results  in 
a listing  of  the  parts  of  the  program,  followed  by  another  part 
query.  This  time,  the  user's  response  (C)  is  1,  indicating  that  he 
wishes  to  read  a data  file.  This  results  in  a prompt  for  the  file 
name,  which  the  user  enters  (D). 

The  program  then  reads  the  beginning  of  the  specified  file  and 
outputs  a header  line  identifying  the  file  with  its  name,  the  run 
number  associated  with  its  creation,  and  the  data  and  time  the  run 
was  conducted.  The  point  defining  the  beginning  of  the  scoring 
interval  is  then  noted  to  the  user,  and  he  is  provided  the 
opportunity  of  changing  it  if  he  so  desires;  here  the  user's 
response  (E)  is  negative,  indicating  that  the  start  point  is 
satisfactory.  The  user  is  then  questioned  as  to  whether  the  stick 
is  to  be  expressed  in  the  recorded  units  of  pounds  or  in  equivalent 

p 

plant  command  units  of  cm,  cm/sec,  or  cm/sec  ; here,  the  user 
indicates  the  latter  (F). 

With  these  preliminaries  completed,  the  user  is  then  able  to 
proceed  to  anv  other  program  part  as  desired;  the  following 
sections  describe  the  possible  options. 

A. 4 .2.2  Parameter  Listing 

The  listing  given  on  the  following  page  illustrates  how  the 
user  can  request  (A)  a complete  printout  of  the  tracking  task 
parameters  associated  with  the  data  being  analyzed.  The  printout 
includes  information  on  the  time  base,  input  signal,  plant,  and 
scoring  and  calibration  parameters.  This  information  effectively 


138 


m 


WHICH  PARTT<0-8> 


PART  i:  File  RF All 

?!  HEADER  LIST 
3:  SCORES 
«S  SPECTRUM 
5!  BE  SC  f CN 
6:  Ot'S  NOISE 
7S  AMP  DIST 
a:  SUMMARY 


WHICH  PAR T t v ( 


iO-a>(sTY''c 

w /■  V" D 

ENTER  PILE  NAME  FOR  INIllTfi  B5 . SAH  J 


FILES  PS .SAH  RUN  NUMBERS  5 DATE  S 18-APR-78  TIMES  3S10S21 

NOMINAL  B 


SCORING  STARTS  AT  POINT 
STICK  MEASURE!!  IN  IB  OR 


501  WANT  TO 
IN  CM'sQ?)—  F 


CHANGE 


i 


1 


139 


WHICH  PARfKO-fn^^—  A 

******************************************* **************** *********** 

**#**!  1ME  BASE  PARAME  TENS***** 

SAMPLE  PER  I OP  ( SECONDS  > --  4.000E-02 

NUMBER  GF  POINTS  IN  RUN  4626 
RUN  DURATION  I SECONDS  ) 1.850EF02 

*****INPUT  SIGNAL  PARAMETERS***** 

TASK  IS  DISTURBANCE  NULLING 

NUMBER  OF  POINTS  IN  INPUT  SIGNAL  409<s 
INPUT  SIGNAL  DURA  1 I ON < SE CONDS > 1 .638E402 

BASE  FREQUENCY (RAD/ST  * 3.835E-02 

PSD  FUNCTION  IS  FIRST  ORDER 


POLF  LOCATION! RAD/SEC ) * 5.000E-0I 
RMS  INPUT  LEVEL  » t.OOOF.+OO 

SPECTRAL  COMPOSITION  OF  INPUT  SIGNAI 


COMP 

HARM 

FREQ 

AMP 

PHASE 

1 

7 

0.27 

0.90 

130.61 

** 

13 

0.50 

0.67 

231 .16 

3 

29 

l.tl 

0.52 

151 . 44 

4 

37 

1.42 

0.34 

177. 9H 

S 

53 

2.03 

0.32 

64.95 

6 

73 

2.80 

0.27 

227.46 

7 

103 

3.95 

0.24 

60.12 

8 

14V 

5. 71 

0 . 20 

113. 29 

9 

211 

8.09 

0.16 

138.96 

10 

293 

11.24 

0.14 

173.67 

11 

419 

16.07 

0.12 

151.26 

12 

587 

22.51 

0.10 

64.69 

13 

839 

32.13 

0.08 

106.44 

(****PL AN  1 PARAMETERS***** 


PLANT  TYPE  ■=  4 
GAIN  = 2.000F.F00 

POLE  <RAD/S)~  -2.000E+00 
TIME  DELAY  = O.OOOE-Ol 

STICK  GAIN  * 5.000E+00 


*****SCORING  PARAMETERS***** 

STICK  WEIGHTING  O.OOOE  01 

SCORING  STARTS  AT  POINT  501 
AND  STOPS  AT  POINT  4596 

*****rAL I BRAT  I ON  PARAMETERS***** 

FIRST  CALIBRATION  LEVH  -t .OOOF+OO 

SECOND  CALIBRATION  l PVT  I 1.0001  TOO 

**********  ********* **  *************** **«**<  * *1  * *******  ***************** 
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augments  the  laboratory  notebook  and  can  be  used  to  precisely 
identify  the  experimental  conditions  under  which  the  tracking  data 
was  collected . 


A. 2. 3 Tracking  Scores 

The  listing  given  on  the  following  page  illustrates  ow  the 
user  can  request  (A)  a printout  of  the  tracking  scores.  The 
program  prompts  the  user  as  to  whether  or  not  he  wishes  the  scores 
calculated  for  the  rates  of  the  tracking  variables;  here  he 
responds  affirmatively  ( B) . The  program  then  sequentially  reads  in 
the  stored  time  history  for  each  variable  and  calculates  its  mean, 
standard  deviation,  and  RMS  value  over  the  scoring  interval.  In 
addition,  since  the  rate  scores  are  requested,  the  signal's  power 
spectrum  is  computed;  this  is  used  in  a frequency  domain 
calculation  of  the  signal's  rate  statistics. 

With  all  four  tracking  signals  so  processed,  the  program  lists 
the  pertinent  scores  as  shown.  In  addition,  the  overall  tracking 
score  (COST)  is  given,  based  on  the  stick  and  display  scores,  and 
the  stick  weighting  factor  (which  is  zero  in  this  case). 

A. 4. 2. 4 Signal  Spectra 

The  listing  just  described  also  illustrates  how  the  user  can 
request  (C)  a printout  of  a signal's  power  spectrum.  His  null 
response  (D)  to  the  signal  query  results  in  a letter  code  listing 
for  each  signal,  followed  by  another  signal  query.  The  user's 
response  at  this  point  (E)  results  in  a printout  of  the  file  header 
information,  followed  by  the  power  spectrum  for  the  stick. 

Since  the  spectrum  is  calculated  at  the  frequencies  contained 
in  the  loop  input  (see  parameter  listing  of  section  A. 4. 2. 2),  the 
first  two  columns  give  the  component  index  (COMP)  and  frequency 
(FREQ).  The  next  two  columns  give  the  correlated  (COR)  and  remnant 
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WHICH  PART*<0 


WANT  RATE  SCORES 


FILESB3.SAH 


RESt^V)-  B 


RUN  number: 


DATE  1 18-APR-78 


time:  3:10:21 


OARBL 

MEAN 

S.D. 

RMS 

INPT 

0.00 

1.00 

1.00 

STCK 

-0.02 

1.59 

1.39 

PLNT 

0.02 

0.44 

0.44 

DISP 

0.02 

0.44 

0.44 

INPTDOT 

0.00 

3.48 

3.48 

STCKDOT 

0.00 

7.96 

7.96 

PLNTDOT 

-0.00 

1.85 

1.85 

DISPPOT 

-0.00 

1.83 

1.85 

COST  » 

0.44 

SIGNAL  CODE  IS!  1 FOR  INPUT 
S STICK 

P Pt  ANT 

D DISPIAY 

C PLANT  COMMAND 


TRY  again: 


(EH 


fileibs.sah 


run  numker:  r.  date:i8-apr-78  iiml:  3:10:21 


SPECTRUM  FOR  STCK 
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REM 
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C7R 

NREM 

1 

0.27 

-2.58 

-43.74 
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-29.58 
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~\ 

2 

0.30 
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-38 • 56 

34.03 

0.07 
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24.47 

2 

3 

1.11 
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-36.21 

31.07 
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-22.05 

18.85 

5 

4 

1.42 

-8.06 

-33.68 

25.62 

-4.27 

-19.51 

15.74 

6 

5 

2.03 

-7.55 

-35.04 

27.49 

-5.99 

-20.88 

14.89 

9 

6 

2.80 

-8.12 

-34.13 

76.02 

-7.79 

-19.97 

17.18 

13 

7 

3.93 

-7.51 

-28.80 

21  .29 

-8.99 

-1 4.04 

5 . 65 

18 

8 

5.71 

-8.57 

-29.34 

20.77 

-11.73 

-15.18 
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75 

9 
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17.52 

-14.92 
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37 

10 
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-19.00 
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51 

11 
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-36. 70 

17.69 

-26.51 

-22.54 

-3.98 

73 

17 

22.31 

-30.01 

-44.97 

11.96 

30.93 

-30.81 

8.12 

102 

13 

32.18 

-35.53 

-47.34 
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-46. 15 

33.  18 

-17.97 
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TOTAL  PWR 

2.53 

COR  PWK 

2.16 
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0.85 
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( REM)  power,  the  former  being  simply  the  signal  power  at  the 
indicated  frequency,  and  the  latter  obtained  by  averaging  the 
signal  power  over  the  neighboring  measurement  frequencies.  The 
power  is  expressed  in  decibels  (dB),and,  in  this  case,  reflects 
stick  power  per  measurement  window,  expressed  in  the  units 
specified  by  the  user  during  the  file  read  procedure  (recall 
section  A. 4. 2.1).  The  next  column  gives  the  signal's 
correl ated- to-remnant  power  ratio  (C/R),  to  provide  the  user  with 
an  indication  of  how  well-correlated  the  signal  is  with  the  loop 
input . 

Since  these  power  measures  reflect  the  width  of  the 
measurement  window,  that  is,  the  frequency  resolution  of  the 
digitized  spectrum,  the  program  also  estimates  the  signal's  power 
spectral  density.  The  next  two  columns  (labelled  COR  and  REM)  give 
the  corresponding  power  spectral  densities  in  terms  of 
power/( radian/second) , with  the  next  to  last  column  (C/R)  giving 
the  associated  C/R  ratio.  The  column  (NREM)  displays  the  number  of 
neighboring  measurement  frequencies  utilized  in  estimating  the 
remnant  power  at  each  correlated  frequency. 

Following  this  spectrum  printout,  are  total  power  measurement 
figures,  for  both  the  correlated  and  remnant  portions  of  the 
signal,  expressed  not  in  dB  but  in  units  associated  with  the  signal 
(in  this  case,  cm  ) . Also,  given  are  the  correlated  and  remnant 
power  ratios,  useful  in  indicating  signal  coherence. 

The  program  then  questions  the  user  as  to  whether  or  not 
another  signal  spectrum  is  desired.  Here,  the  user  response  is 
negative  (F);  had  it  been  affirmative,  the  user  would  have  been 
prompted  for  the  appropriate  signal  code,  and  the  above  process 
repeated . 
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A. *4. 2.5  Describing  Functions 


The  listing  given  on  the  following  page  illustrates  how  the 
user  can  request  (A)  a describing  function  printout  relating  any 
one  loop  variable  to  any  other.  The  program  prompts  the  user  for 
the  symbol  of  the  signal  to  be  associated  with  the  describing 
function  numerator  (B),  and  denominator  (C).  In  this  example,  the 
user  requests  a computation  of  the  display-to-stick  describing 
function . 

The  program  responds  with  the  file  header  information  followed 
by  a describing  function  identifier  (DFCN),  followed  by  the 
computed  gain  and  phase  at  each  loop  input  signal  frequency.  As 
before,  the  frequency  is  in  radians/second;  gain  and  phase  are  in 
dB  and  degrees,  respectively. 

The  program  then  questions  the  user  as  to  whether  or  not 
another  describing  function  is  desired.  Here,  the  response  is 
negative  (D);  had  it  been  affirmative,  the  user  would  have  been 
prompted  for  the  appropriate  numerator  and  denominator  signal 
codes,  and  the  above  process  repeated. 

A. 4. 2. 6 Observation  Noise 

The  listing  just  described  also  illustrates  how  the  user  can 
request  (E)  a printout  of  the  computed  observation  noise  associated 
with  the  subject's  tracking  performance. 

The  program  responds  with  the  file  header  information, 
followed  by  a data  identifier,  followed  by  the  observation  noise 
power  computed  at  each  frequency  contained  in  the  loop  input.  As 
before  the  frequency  is  in  radians/second,  and  the  power  is  in  dB. 
The  column  labelled  POSN  gives  the  noise  related  to  the  display 
displacement,  and  the  column  labelled  RATE  gives  the  noise  related 
to  display  rate. 
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COMP 

FREQ 

GAIN 

PHASE 

1 

0.27 

16.71 

-36.23 

O 

0.50 

13.53 

-31.57 

3 

1.11 

10.84 

-28.85 

A 

1.42 

10.03 

-30.58 

s 

2.03 

9.40 

-32.87 

6 

2.80 

9.35 

-34.99 

7 

3.95 

8.92 

-40.96 

8 

5.71 

9.51 

-49.04 

9 

8.09 

10.24 

-62.98 

10 

11.24 

12.18 

-87.93 

11 

16.07 

18.33 

-125.62 

12 

22.51 

20.38 

-266.02 

13 

32.18 

13.19 

-392.59 

ANOTHER  DESCRIBING  FUNCTION 


WHICH  PART7 ( 0-8 


E 


file’.bs.sah  run  number:  5 date: is-afr-78  time:  3:10:21 

OBSERVATION  NOISE 


COMP 

FREQ 

POSN 

RATE 

1 

0.27 

-23.94 

-47.74 

2 

0.50 

-19.59 

-38.01 

3 

1.11 

-19.63 

-31 .09 

4 

1.42 

-18.44 

-27.78 

5 

2.03 

-22.27 

-28.48 

6 

2.80 

-23.85 

-27.29 

7 

3.95 

-22.43 

-22.88 

8 

5.71 

-26.12 

-23.36 

9 

8.09 

-27.42 

-21 .o3 

10 

11.24 

-32.40 

-23.76 

11 

16.07 

-36.19 

-24.45 

12 

22.51 

-37.87 

-23.20 

13 

32.18 

-39.21 

-21.44 

A. 4. 2.7  Amplitude  Distribution 

A. 4.2. 8 Summary  Calculations  and  File  Writing 

The  listing  given  on  the  following  page  illustrates  how  the 
user  can  request  (A)  a printout  summarizing  the  fundamental  human 
operator  measurements  made  possible  by  the  Performance  Analyzer. 
The  user  is  first  prompted  as  to  whether  or  not  a new  input  file  is 
to  be  read  in.  The  user's  negative  response  ( B)  is  interpreted  by 
the  program  as  a request  that  summary  calculations  be  conducted 
with  the  currently  open  file.  Had  the  user  responded 
affirmatively,  a new  file  would  have  been  read  in  and  processed. 

After  prompting,  the  user  indicates  that  the  summary  data  is 
to  be  displayed  on  the  terminal  (Cl,  and  to  be  stored  on  a disc 
file  (D) . This  results  in  a prompt  for  the  file  name  to  be  used 
for  summary  data  storage,  and  the  user  responds  with  the  file  name 
(E).  The  program  then  lists  the  file  header  information 
identifying  the  file  containing  the  tracking  histories,  and  then 
lists  the  tracking  scores  for  the  four  loop  variables.  Also  given 
are  the  scores  for  the  stick  rate  and  the  overall  tracking  score. 
These  scores  are  followed  bv  the  pertinent  frequency  domain 
information,  consisting  of  the  subject's  calculated  gain  and  phase 
(stick/display)  and  his  observation  noise,  referred  to  display 
position  and  rate.  The  first  blank  column  is  available  to  flag  low 
C/R  ratios  in  either  the  stick  or  display  spectra,  and  the  last 
column  is  available  to  flag  possibly  invalid  observation  noise 
calculations.  Additional  details  concerning  the  score,  describing 
function,  and  observation  noise  calculations  are  to  be  found  in 
sections  A. 4. 2. 3,  A. 4. 2. 5,  and  A. 4.2. 6,  respectively. 

Following  this  summary  printout,  the  program  writes  the 
corresponding  information  on  the  output  file  designated  earlier  by 
the  user  (TEMP.DAT).  This  allows  the  user  to  save  the  summary 
calculations  for  possible  later  analysis  and/or  ensemble  averaging. 
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WHICH  PART? < 0 


NEW  INPUT  FILE? 


OUTPUT  ON  TT?.MY 


OUTPUT  ON  DK?:(Y 


-8>(ffT)—  A 

e?j(n%—  B 
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7 

3.95  3.92 

-40.96 

-22.43 

-22.89 

8 
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9 
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10 
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-23.76 

11 
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-24.45 

12 
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-23.20 

13 
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A. 5.  INTERFACE  MODULES  AND  REAL-TIME  HARDWARE 


This  section  provides  a brief  description  of  the  input  and 
output  interface  modules  indicated  earlier  in  figure  A. 2 of  section 
A. 2.  Also  given  here  are  brief  summaries  of  the  performance 
specifications  of  the  digital  computer  real-time  hardware  used  to 
implement  the  Performance  Analyzer. 

A. 5.1  Input  Interface 

The  input  interface  consists  of  a force  stick  in  cascade  with 
a noise-suppressing  low-pass  filter. 

The  force  stick  used  is  a Model  *435  Hand  Control  manufactured 
by  Measurement  Systems,  Inc.  of  Norwalk,  Conn.  When  driven  by  an 
AC  source,  the  stick  provides  a DC  voltage  linearly  proportional  to 
the  force  applied  to  the  handle.  Currently,  the  stick  is  driven  by 
an  18.4  volt  (RMS)  source  at  1500  Hz.  With  a modified  extra-length 
handle,  the  stick  provides  approximately  0.83  volts  per  pound  force 
deflection . 

The  low-pass  filter  is  a second-order  non-unity  gain 
Butterworth,  having  a break  frequency  of  12.5  Hz  and  a damping 
ratio  of  0.707.  The  break  frequency  was  chosen  to  equal  half  the 
25  Hz  sampling  rate  (Tgs40  msec)  used  in  the  tracking  experiments 
conducted  with  the  Performance  Analyzer.  The  DC  filter  gain  was 
set  at  4.85  volt/volt,  to  provide  an  overall  force-to- vol tage  gain 
of  4 volt/pound. 

A. 5. 2 Output  Interface 

The  output  interface  consists  of  a Type  503  rack-mounted 
cathode  rav  oscilloscope  manufactured  by  Tektronix,  Inc.  of 
Beaverton,  Oregon.  The  vertical  channel  is  adjusted  to  20  volt/cm, 
and  is  driven  by  the  18.4  volt  RMS  source  used  to  drive  the  stick. 
The  horizontal  channel  is  adjusted  to  one  volt/cm,  and  is  driven  by 
the  D-to-A  channel  used  to  interface  with  the  Performance  Analyzer 
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software.  This  combination  of  vertical  and  horizontal  inputs 
results  in  a vertical  bar  display,  having  a height  of  2.6  cm,  and  a 
horizontal  displacement  proportional  to  the  D-to-A  output  voltage. 

A. 5. 3 Real-Time  Hardware 

The  real-time  hardware  used  to  implement  the  Performance 
Analyzer  consists  of  a real-time  clock  and  A-to-D  and  D-to-A 
converters,  all  manufactured  by  Digital  Equipment  Corp.  of  Maynard, 
Mass  . 

The  clock  is  a DEC  KW1 1-K  dual  programmable  real-time  clock, 
and  is  used  in  a repeated  interval  mode  for  timing  information  by 
the  Performance  Analyzer.  A variety  of  clock  functions  are  called 
for  by  the  Analyzer  software,  including  starting  and  stopping  the 
deck,  setting  its  base  rate,  initializing  a ’'stop  watch”  counter, 
and  reading  the  counter's  contents.  These  functions  are  currently 
performed  by  a set  of  clock  handler  routines  comprising  one  of  the 
systems's  resident  utility  libraries. 

The  A-to-D  converter  is  a DEC  ADI 1-K  anal og- to-d ig ital 
converter,  which  is  a 12-bit  16-channel  converter  currently 
configured  for  bipolar  5 volt  inputs.  The  22  microsecond 
conversion  time  is  an  insignificant  fraction  of  typical  sample 
times  used  during  Performance  Analyzer  operation  (Ts=40  msec)  , and 
the  12-bit  resolution  provides  for  a quantization  error  on  the 
order  of  one  millivolt,  a level  well  below  expected  signal 
transmission  noise  levels.  The  A-to-D  handler  utilized  bv  the 
Performance  Analyzer  provides  a check  for  conversion  errors,  and 
maintains  control  of  the  system  software  during  a conversion. 

The  D-to-A  converter  is  a DEC  AA11-K  d igital-to-analog 
converter,  which  is  a 12-bit  4-channel  converter  currently 
configured  for  bipolar  5 volt  outputs.  The  conversion  time  is  on 
the  order  of  5 microseconds,  again  insignificant  with  respect  to 


typical  sample  times  used;  quantization  errors  are  the  same  as 
those  of  the  A-to-D  converter. 

A. 5. 4 Interface  Module  Diagrams 

Figure  A.1  summarizes  the  input/output  interfaces  just 
described,  and  illustrates  the  interconnection  between  the  A-to-D 
converter,  D-to-A  converter,  stick  and  display.  For  completeness, 
figure  A. 4 illustrates  in  block  diagram  form  the  signal  processing 
provided  by  the  Performance  Analyzer  software. 


Figure  A. 3a:  Input  Interface 


Appendix  B:  Task  Dynamics 


r 


A series  of  input-output  measurements  were  made  on  the 
Performance  Analyzer  to  validate  the  proper  implementation  of 
the  desired  task  dynamics.  Measurements  were  made  in  "autopilot 
mode",  which  consisted  of  connecting  the  display  signal  line 
directly  to  the  input  of  the  Butterworth  filter  used  for 
conditioning  the  stick  signal  (see  figures  A. 3 and  A. 4 of 
appendix  A) , the  stick  signal  having  been  disconnected  for 
these  measurements.  In  this  way  the  human  operator  was  eliminated 
from  the  task,  and  loop  stabilization  was  provided  by  the  display- 
to- filter  feedback. 

An  earlier  calibration  was  used  to  ensure  that  one  volt  of 
display  signal  resulted  in  one  centimeter  of  display  displacement. 
In  addition,  the  calibration  resulted  in  one  volt  of  stick  signal 
being  interpreted  as  1.21  pounds  force.  Since  the  "autopilot"  used 
in  these  measurements  had  unity  voltage  gain,  its  gain,  as  seen 
by  the  Performance  Analyzer  was  1.212  lbf/cm.  To  provide  negative 
feedback  and  avoid  saturation,  the  stick  gain  Kg  (see  figure  A. 4 of 
appendix  A)  was  chosen  to  be  -3  cm/lbf,  opposite  in  sign  and 
of  smaller  magnitude  than  the  5 cm/lbf  value  used  in  the  human 
operator  runs.  The  equivalent  autopilot  gain  was  thus  -3.64  cm/ cm, 
or  11.23  dB,  if  the  sign  is  discounted. 

A tracking  run  was  conducted  with  the  autopilot,  using  the 
same  plant  dynamics  and  loop  input  disturbance  function  as  were 
used  in  the  human  operator  tracking  runs.  In  addition,  the  same 
40  msec  sample  time  was  used,  to  provide  a control  for  sample- 
and-hold  effects.  After  the  run,  the  Performanace  Analyzer  was 
used  to  calculate  the  describing  function  relating  plant  command 
pc  to  plant  output  p , as  a check  on  the  unstable  plant  dynamics 
specified  in  the  tracking  task. 
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Figure  B.l  shows  the  measurements  obtained,  along  with  the 
theoretical  curves  associated  with  the  specified  2/(s-2)  dynamics. 

Except  for  the  highest  measurement  frequency  where  the  gain  drops 
1.5  dB  below  that  desired,  the  measured  gain  and  phase  follow  the 
continuous  curves  quite  closely,  and  thus  validate  the  internal 
plant  dynamics  implemented  on  the  Performance  Analyzer. 

Reference  to  figure  A. 4 of  appendix  A shows,  however,  that 
this  measurement  is  across  the  digital  filter  D(z)  used  to  implement 
the  plant  dynamics.  Thus,  it  is  internal  to  the  Performance 
Analyzer  and  takes  no  account  of  the  overall  input-output  delays 
introduced  by  the  computation  times  and  sampling  holds  characterizing 
the  digital  system.  These  effects  can  be  measured  quite  simply, 
however,  by  using  the  Performance  Analyzer  to  calculate  the 
autopilot  describing  function,  relating  effective  stick  output 
to  display  input.  If  no  distortion  is  introduced  by  the  digital 
system,  then  one  would  expect  the  autopilot  measurement  to  simply 
reflect  the  autopilot  gain  in  cascade  with  the  Butterworth  filter 
dynamics,  since  the  filter  was  used  to  condition  the  autopilot 
signal . 

Figure  B.2  shows  the  gain  and  phase  measured  for  the  autopilot, 
plotted  linearly  against  frequency.  The  gain  measurements  are 
quite  close  to  the  expected  11.23  dB,  with  less  than  0.2  dB  drop 
at  the  highest  frequency  of  32  rad/sec.  Since  the  Butterworth 
filter,  with  its  78  rad/sec  break  frequency,  contributes  approximately 
0.1  dB  drop  at  32  rad/sec,  one  can  conclude  that  the  Performance 
Analyzer  contributes  effectively  no  amplitude  distortion. 

Phase  distortion  is  introduced,  however,  as  is  clear  from 
the  increasing  lag  with  frequency  seen  on  the  figure.  The 
measurements  made  on  the  zero-phase  autopilot  are  indicated  by  the 
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open  circles,  and  can  be  well-modelled  by  a simple  dead-time. 

The  straight  line  fitting  the  data  indicates  an  effective  dead- 
time of  approximately  32  msec,  a delay  which  is  due  to  both  the 
Butterworh  filter  dynamics  and  the  digital  implementation  of  the 

I Performance  Analyzer.  This,  then,  is  a delay  which  the  human 

operator  sees  while  tracking  the  display,  in  addition  to  the  dynamic 
lags  introduced  by  the  plant  transfer  function  of  figure  B.l. 

Human  operator  measurements  made  with  the  Performance  Analyzer  must 
thus  be  compensated  by  the  addition  of  a 32  msec  lead,  to  arrive 
at  the  true  operator  describing  function.  As  might  be  expected, 
this  32  msec  figure  is  a function  of  sample  rate  and  stick  filter 
break  frequency,  and  thus  is  only  appropriate  for  the  single 
configuration  described  here. 

The  phase  shift  due  to  the  stick  filter  above  can  be  computed 
as  a function  of  frequency,  based  on  the  parameter  values  given 
in  figure  A. 3 of  appendix  A.  If  this  phase  shift  is  then  added  to 
the  measured  phase  of  figure  B.2,  the  phase  values  indicated  by 
the  closed  circles  result.  The  straight-line  fit  through  this  data 
set  indicates  a dead-time  of  approximately  13  msec,  and  can  be 
attributed  to  the  Performance  Analyzer's  digital  processing  delays, 
when  operating  with  a sample  period  of  40  msec. 
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APPENDIX  C:  TABULATED  EXPERIMENTAL  RESULTS 


This  appendix  contains  tabulated  data  to  supplement  the 
discussion  given  in  Chapter  5.  Tables  C.l  and  C.2  give  across- 
subject  means  and  standard  deviations  of  tracking  scores  obtained 
during  the  initial  and  intermediate  training  phases  of  the 
experiment,  respectively.  The  latter  table  also  gives  a break- 
down with  tracking  condition. 

Table  C.3  gives  across-sub ject  means  and  standard  deviations 
of  performance  scores  obtained  during  the  control  phase  of  the 
experiment,  for  each  condition  tested.  Significance  levels  are 
also  given  for  differences  between  conditions,  based  on  indi- 
vidual subject  paired  differences;  levels  greater  than  0.05  are 
considered  not  significant  (NS) . Similar  information  for  the 
frequency  response  measures  are  given  in  Tables  C.4  (amplitude 
ratio),  C.5  (phase  shift),  and  C.6  (remnant  ratio). 

The  mean  phase  shifts  of  Table  C.5  include  a 32  msec  lead 
to  account  for  delays  introduced  by  the  Performance  Analyzer. 

In  addition,  the  mean  phase  shifts  associated  with  the  time 
delay  condition  include  an  additional  40  msec  lag  to  simulate 
an  operator  delay,  so  that  these  phase  means  include  a net  8 
msec  lag. 
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TABLE  C.l:  TRACKING  SCORE  VS.  RUN  NUMBER  (initial  training  phase) 


Score  (cm)  Score  (cm) 

Standard  Standard 


Run 

Mean 

Deviation 

Run 

Mean 

Deviation 

1 

1.54 

.658 

25 

.60 

.103 

2 

1.29 

.515 

26 

.57 

.081 

3 

1.12 

.526 

27 

.52 

.080 

4 

1.14 

.692 

28 

.53 

.089 

5 

.94 

.317 

29 

.53 

.086 

6 

.96 

.434 

30 

.55 

.082 

7 

.81 

.311 

31 

.54 

.104 

8 

.80 

.220 

32 

.52 

.063 

9 

.75 

.174 

33 

.52 

.068 

10 

.64 

.136 

34 

.57 

.126 

11 

.74 

.183 

35 

.56 

.094 

12 

.71 

.174 

36 

.53 

.060 

13 

.71 

.226 

37 

.54 

.043 

14 

.62 

.137 

38 

.51 

.080 

15 

.64 

.234 

39 

.49 

.088 

16 

.63 

.149 

40 

.46 

.056 

17 

.66 

.224 

41 

.46 

.068 

18 

.65 

.147 

42 

.46 

.064 

19 

.67 

.194 

43 

.46 

.076 

20 

.62 

.129 

44 

.50 

.091 

21 

.65 

.152 

45. 

.47 

.059 

22 

.56 

.107 

46 

.47 

.059 

23 

.62 

.117 

47 

.49 

.058 

24. 

.57 

.109 

48 

.49 

.080 
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TABLE  C. 2 : 


TRACKING  SCORE  VS.  RUN  NUMBER  AND  CONDITION 
(intermediate  training  phase) 


! 


TABLE  C.3:  PERFORMANCE  SCORES 


DESCRIBING  FUNCTION  AMPLITUDE  RATIOS  (dB) 
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TABLE  C . 5 : DESCRIBING  FUNCTION  PHASE  SHIFTS  (deg. 
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TABLE  C . 6 : NORMALIZED  REMNANT  RATIOS  (dB) 
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